]>
git.decadent.org.uk Git - dak.git/log
Mark Hymers [Wed, 23 Mar 2011 14:36:28 +0000 (14:36 +0000)]
Work with service mode
Signed-off-by: Mark Hymers <mhy@debian.org>
Torsten Werner [Wed, 23 Mar 2011 08:54:45 +0000 (09:54 +0100)]
Configure and test more mapper relationships.
- DBBinary and DBSource have relationships to BinaryMetadata and SourceMetadata.
- Both have an associationproxy to hide the details.
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Tue, 22 Mar 2011 22:02:25 +0000 (23:02 +0100)]
Add Metadata classes + tests.
+ some cleanup of dbconn.py
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Tue, 22 Mar 2011 20:09:06 +0000 (21:09 +0100)]
Merge branch 'contents' of ftp-master.debian.org:public_html/dak into contents
Torsten Werner [Tue, 22 Mar 2011 20:05:58 +0000 (20:05 +0000)]
Replace DBConn.reset() by something more robust.
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Tue, 22 Mar 2011 19:18:14 +0000 (19:18 +0000)]
Fix database tests.
Signed-off-by: Torsten Werner <twerner@debian.org>
Joerg Jaspert [Tue, 22 Mar 2011 18:32:32 +0000 (19:32 +0100)]
gps
move stable/oldstable into gps too, so we only generate packages/sources
from there, ever
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Joerg Jaspert [Tue, 22 Mar 2011 18:01:28 +0000 (19:01 +0100)]
license
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Torsten Werner [Tue, 22 Mar 2011 16:48:13 +0000 (16:48 +0000)]
Contents: replace multithreading by multiprocessing.
That avoids the global interpreter lock of CPython.
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Tue, 22 Mar 2011 15:50:39 +0000 (16:50 +0100)]
Merge branch 'contents'
Torsten Werner [Tue, 22 Mar 2011 15:42:58 +0000 (15:42 +0000)]
Add more logging to contents generate.
Signed-off-by: Torsten Werner <twerner@debian.org>
Mark Hymers [Tue, 22 Mar 2011 15:07:29 +0000 (15:07 +0000)]
Merge remote branch 'mhy/master' into merge
Signed-off-by: Mark Hymers <mhy@debian.org>
Mark Hymers [Tue, 22 Mar 2011 15:06:04 +0000 (15:06 +0000)]
Add Built-Using support
Signed-off-by: Mark Hymers <mhy@debian.org>
Torsten Werner [Tue, 22 Mar 2011 14:54:11 +0000 (14:54 +0000)]
Contents: ignore errors when removing destination file.
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Tue, 22 Mar 2011 14:47:09 +0000 (14:47 +0000)]
Do no longer re-exec dak contents.
We just call DBConn().reset() in the child process now.
Signed-off-by: Torsten Werner <twerner@debian.org>
Joerg Jaspert [Tue, 22 Mar 2011 14:31:15 +0000 (15:31 +0100)]
ries sync
no longer push or name anything merkel
sync ries differently depending on if its called with sync or pool
that way cron.hourly does a small sync while cron.dinstall a full one triggers
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Joerg Jaspert [Tue, 22 Mar 2011 13:50:12 +0000 (14:50 +0100)]
shutup
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Torsten Werner [Tue, 22 Mar 2011 13:18:55 +0000 (13:18 +0000)]
Contents: use the new string_agg() function of Pg9.
Signed-off-by: Torsten Werner <twerner@debian.org>
Joerg Jaspert [Tue, 22 Mar 2011 13:09:55 +0000 (14:09 +0100)]
Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak
* 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak:
Fix typo
Improve db conf handling
Commit uncommitted fix
Patch in support for service= DSN syntax
Joerg Jaspert [Tue, 22 Mar 2011 13:09:49 +0000 (14:09 +0100)]
add the syncscript for ries. also call it hourly
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Mark Hymers [Tue, 22 Mar 2011 10:56:55 +0000 (10:56 +0000)]
Don't remove sources which are referenced by other binaries
Signed-off-by: Mark Hymers <mhy@debian.org>
Mark Hymers [Tue, 22 Mar 2011 10:42:39 +0000 (10:42 +0000)]
Fix typo
Signed-off-by: Mark Hymers <mhy@debian.org>
Mark Hymers [Tue, 22 Mar 2011 10:41:51 +0000 (10:41 +0000)]
Merge remote branch 'mhy/master' into merge
Signed-off-by: Mark Hymers <mhy@debian.org>
Mark Hymers [Tue, 22 Mar 2011 10:41:17 +0000 (10:41 +0000)]
Improve db conf handling
Signed-off-by: Mark Hymers <mhy@debian.org>
Mark Hymers [Tue, 22 Mar 2011 10:21:58 +0000 (10:21 +0000)]
Commit uncommitted fix
Signed-off-by: Mark Hymers <mhy@debian.org>
Mark Hymers [Tue, 22 Mar 2011 10:20:18 +0000 (10:20 +0000)]
Merge remote branch 'mhy/master' into merge
Signed-off-by: Mark Hymers <mhy@debian.org>
Mark Hymers [Tue, 22 Mar 2011 10:18:57 +0000 (10:18 +0000)]
Patch in support for service= DSN syntax
Signed-off-by: Mark Hymers <mhy@debian.org>
Joerg Jaspert [Tue, 22 Mar 2011 10:14:10 +0000 (11:14 +0100)]
add ries
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Mark Hymers [Mon, 21 Mar 2011 21:45:58 +0000 (21:45 +0000)]
Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak
Mark Hymers [Mon, 21 Mar 2011 21:45:43 +0000 (21:45 +0000)]
Merge remote branch 'mhy/master' into merge
Signed-off-by: Mark Hymers <mhy@debian.org>
Mark Hymers [Mon, 21 Mar 2011 21:38:29 +0000 (21:38 +0000)]
Use PK at twerners suggestion
Signed-off-by: Mark Hymers <mhy@debian.org>
Mark Hymers [Mon, 21 Mar 2011 21:35:47 +0000 (21:35 +0000)]
set db schema version
Signed-off-by: Mark Hymers <mhy@debian.org>
Torsten Werner [Mon, 21 Mar 2011 21:26:21 +0000 (22:26 +0100)]
Add method reset() to class DBConn() and test it.
Signed-off-by: Torsten Werner <twerner@debian.org>
Mark Hymers [Mon, 21 Mar 2011 20:55:49 +0000 (20:55 +0000)]
Merge remote branch 'ftpmaster/master'
Mark Hymers [Mon, 21 Mar 2011 20:26:30 +0000 (20:26 +0000)]
First pass at P/S database schema
Signed-off-by: Mark Hymers <mhy@debian.org>
Mark Hymers [Mon, 21 Mar 2011 20:18:26 +0000 (20:18 +0000)]
First pass at extra_src_references table
Signed-off-by: Mark Hymers <mhy@debian.org>
Torsten Werner [Mon, 21 Mar 2011 20:07:33 +0000 (21:07 +0100)]
Add table src_contents.
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Mon, 21 Mar 2011 19:44:39 +0000 (20:44 +0100)]
Merge branch 'master' of ftp-master.debian.org:/srv/ftp.debian.org/git/dak
Torsten Werner [Mon, 21 Mar 2011 19:43:29 +0000 (20:43 +0100)]
Add logging to update-db including transaction id.
Signed-off-by: Torsten Werner <twerner@debian.org>
Mark Hymers [Mon, 21 Mar 2011 19:23:58 +0000 (19:23 +0000)]
Merge remote branch 'mhy/master' into merge
Signed-off-by: Mark Hymers <mhy@debian.org>
Torsten Werner [Mon, 21 Mar 2011 19:18:53 +0000 (20:18 +0100)]
Really update to DB version #44.
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Mon, 21 Mar 2011 17:21:16 +0000 (18:21 +0100)]
Merge branch 'master' into dbtests
Conflicts:
dak/update_db.py
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Mon, 21 Mar 2011 17:14:26 +0000 (18:14 +0100)]
Rename DB upgrade #43 to #44.
bad, bad mhy...
Signed-off-by: Torsten Werner <twerner@debian.org>
Joerg Jaspert [Mon, 21 Mar 2011 15:56:26 +0000 (16:56 +0100)]
make check_overrides not delete stuff thats not at least 14 days old
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Mark Hymers [Mon, 21 Mar 2011 15:32:10 +0000 (15:32 +0000)]
Don't contradict ourselves within 2 lines
Signed-off-by: Mark Hymers <mhy@debian.org>
Mark Hymers [Mon, 21 Mar 2011 15:06:30 +0000 (15:06 +0000)]
add mvo quote
Signed-off-by: Mark Hymers <mhy@debian.org>
Joerg Jaspert [Mon, 21 Mar 2011 13:17:57 +0000 (14:17 +0100)]
Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak
* 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak:
Prepare for pg 9.0
Joerg Jaspert [Mon, 21 Mar 2011 13:17:45 +0000 (14:17 +0100)]
gps
change the way we run the pool once more, hopefully being less bad now. also, this tool is EOL somehwat, so who cares how bad it looks
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Mark Hymers [Mon, 21 Mar 2011 11:44:59 +0000 (11:44 +0000)]
Prepare for pg 9.0
Signed-off-by: Mark Hymers <mhy@debian.org>
Joerg Jaspert [Mon, 21 Mar 2011 11:32:16 +0000 (12:32 +0100)]
gps timeout/error handling
return the result of the a-f call.
additionally get it using a timeout.
when either a TimeoutError occurs or the result is non-zero, complain about it
and then exit non-zero, so dinstall could stop.
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Joerg Jaspert [Mon, 21 Mar 2011 11:28:45 +0000 (12:28 +0100)]
mhy, das kruemelmonster
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Joerg Jaspert [Mon, 21 Mar 2011 09:39:05 +0000 (10:39 +0100)]
database backup foo
no longer dump the full database on each dinstall run. instead we just
save a transaction id. DSA is doing the backup dumps in whatever cycle,
and keeping the WAL stuff around, so we can tell them to get us to
whatever point in our db which we might need, ie. the specific transaction
id.
as those are just 9 bytes per file, and not 4gigabytes, we now also save
wth we have been before and after unchecked runs, which makes our backup
be nearer to actual filesystem state than ever.
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Joerg Jaspert [Mon, 21 Mar 2011 08:43:58 +0000 (09:43 +0100)]
remove pgdakdev
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Torsten Werner [Mon, 21 Mar 2011 08:24:12 +0000 (09:24 +0100)]
Remove old contents code and tables.
Signed-off-by: Torsten Werner <twerner@debian.org>
Joerg Jaspert [Sun, 20 Mar 2011 23:46:00 +0000 (00:46 +0100)]
parse scriptvars, not . vars
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Torsten Werner [Sun, 20 Mar 2011 23:01:40 +0000 (00:01 +0100)]
Fix a comment in dbconn.py.
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Sun, 20 Mar 2011 22:30:46 +0000 (23:30 +0100)]
Fix copy-paste error.
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Sun, 20 Mar 2011 22:03:30 +0000 (23:03 +0100)]
Merge branch 'master' of ftp-master.debian.org:/srv/ftp.debian.org/git/dak
Torsten Werner [Sun, 20 Mar 2011 22:02:34 +0000 (23:02 +0100)]
Ignore more warnings from python-apt.
Signed-off-by: Torsten Werner <twerner@debian.org>
Joerg Jaspert [Sat, 19 Mar 2011 14:05:06 +0000 (15:05 +0100)]
Merge branch 'master' of ssh://franck.debian.org/srv/ftp.debian.org/git/dak
* 'master' of ssh://franck.debian.org/srv/ftp.debian.org/git/dak:
Add filename de-duplication to contents scanning.
Joerg Jaspert [Sat, 19 Mar 2011 14:04:59 +0000 (15:04 +0100)]
updates
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Torsten Werner [Tue, 15 Mar 2011 18:14:47 +0000 (19:14 +0100)]
Merge branch 'dbtests'
Torsten Werner [Tue, 15 Mar 2011 17:58:39 +0000 (18:58 +0100)]
Add filename de-duplication to contents scanning.
Signed-off-by: Torsten Werner <twerner@debian.org>
Julien Cristau [Mon, 14 Mar 2011 21:48:57 +0000 (22:48 +0100)]
byhand-di: allow YYYYMMDD+<suite>x version numbers
To: debian-dak@lists.debian.org
Cc: Julien Cristau <jcristau@debian.org>, Otavio Salvador <otavio@ossystems.com.br>
Date: Mon, 14 Mar 2011 14:11:39 +0100
X-Spam-Status: No, score=-1.775 tagged_above=-99 required=4.6 tests=[BAYES_00=-1.9, SARE_SUB_OBFU_OTHER=0.135, T_RP_MATCHES_RCVD=-0.01] autolearn=no
X-Mailer: git-send-email 1.7.2.5
Resent-From: debian-dak@lists.debian.org
stable updates need to have a higher version than binNMUs, but
YYYYMMDD<suite>x << YYYYMMDD+b1.
Signed-off-by: Julien Cristau <jcristau@debian.org>
Cc: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Torsten Werner [Mon, 14 Mar 2011 21:07:40 +0000 (22:07 +0100)]
Break hardlink when writing Contents files.
Signed-off-by: Torsten Werner <twerner@debian.org>
Joerg Jaspert [Sat, 12 Mar 2011 14:05:44 +0000 (15:05 +0100)]
we do want -n1, not -n16. we want as many parallel xz as we can
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Joerg Jaspert [Sat, 12 Mar 2011 09:51:42 +0000 (10:51 +0100)]
dinstall - mirror
exclude the temporary files from apt-ftparchive (*.new), and delete excluded
files too.
Should keep the mirrors happy, cos those files arent readable for others.
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Torsten Werner [Fri, 11 Mar 2011 22:13:28 +0000 (23:13 +0100)]
Handle empty packages in 'dak contents scan'.
Signed-off-by: Torsten Werner <twerner@debian.org>
Mark Hymers [Fri, 11 Mar 2011 20:27:40 +0000 (20:27 +0000)]
allow squeeze-updates to be reset
Signed-off-by: Mark Hymers <mhy@debian.org>
Torsten Werner [Fri, 11 Mar 2011 19:30:22 +0000 (20:30 +0100)]
Switch to 'dak contents generate'.
It replaces APT::FTPArchive::Contents.
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Thu, 10 Mar 2011 22:07:45 +0000 (23:07 +0100)]
Enable the new contents generation for experimental.
... as a first test
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Thu, 10 Mar 2011 21:52:57 +0000 (22:52 +0100)]
Write full section to Contents files.
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Wed, 9 Mar 2011 08:15:00 +0000 (09:15 +0100)]
Merge branch 'master' into dbtests
Torsten Werner [Wed, 9 Mar 2011 08:14:05 +0000 (09:14 +0100)]
Add workaround for duplicate filenames in contents.
Signed-off-by: Torsten Werner <twerner@debian.org>
Joerg Jaspert [Tue, 8 Mar 2011 21:57:15 +0000 (22:57 +0100)]
Merge branch 'master' of ssh://franck.debian.org/srv/ftp.debian.org/git/dak
* 'master' of ssh://franck.debian.org/srv/ftp.debian.org/git/dak:
bugfixes
Contents: support override suites.
Joerg Jaspert [Tue, 8 Mar 2011 21:57:08 +0000 (22:57 +0100)]
dinstall - use xz for pg dumps, and use a | xargs instead of while
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Torsten Werner [Tue, 8 Mar 2011 21:47:19 +0000 (22:47 +0100)]
Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak
Torsten Werner [Tue, 8 Mar 2011 21:44:26 +0000 (21:44 +0000)]
bugfixes
Signed-off-by: Torsten Werner <twerner@debian.org>
Joerg Jaspert [Tue, 8 Mar 2011 21:35:50 +0000 (22:35 +0100)]
Merge branch 'master' of ssh://franck.debian.org/srv/ftp.debian.org/git/dak
* 'master' of ssh://franck.debian.org/srv/ftp.debian.org/git/dak:
Use 'dak contents generate_helper'.
bugfixes
Add a new subcommand 'generate_helper' to 'dak contents'.
Switch to multiprocessing module and reap child gzips.
Joerg Jaspert [Tue, 8 Mar 2011 21:35:32 +0000 (22:35 +0100)]
dinstall - dont recompress .xz files
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Torsten Werner [Tue, 8 Mar 2011 21:29:43 +0000 (22:29 +0100)]
Contents: support override suites.
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Tue, 8 Mar 2011 20:57:17 +0000 (21:57 +0100)]
Merge branch 'master' into dbtests
Torsten Werner [Tue, 8 Mar 2011 20:55:38 +0000 (20:55 +0000)]
Use 'dak contents generate_helper'.
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Tue, 8 Mar 2011 20:13:06 +0000 (20:13 +0000)]
bugfixes
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Tue, 8 Mar 2011 19:59:26 +0000 (20:59 +0100)]
Add a new subcommand 'generate_helper' to 'dak contents'.
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Tue, 8 Mar 2011 18:39:59 +0000 (19:39 +0100)]
Switch to multiprocessing module and reap child gzips.
Signed-off-by: Torsten Werner <twerner@debian.org>
Joerg Jaspert [Mon, 7 Mar 2011 22:31:03 +0000 (23:31 +0100)]
And make the whitespace something useful
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Joerg Jaspert [Mon, 7 Mar 2011 22:30:33 +0000 (23:30 +0100)]
Fixup the regex
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Joerg Jaspert [Mon, 7 Mar 2011 21:47:49 +0000 (22:47 +0100)]
goodbye comments
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Joerg Jaspert [Mon, 7 Mar 2011 21:47:10 +0000 (22:47 +0100)]
during unchecked no longer recreate testing indices, only do them when importing new data there
Signed-off-by: Joerg Jaspert <joerg@debian.org>
Torsten Werner [Mon, 7 Mar 2011 20:47:02 +0000 (21:47 +0100)]
Scan 10.000 files per unchecked run.
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Mon, 7 Mar 2011 20:46:00 +0000 (21:46 +0100)]
Include all files (except directories) in bin_contents table.
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Sun, 6 Mar 2011 22:31:46 +0000 (23:31 +0100)]
Merge branch 'dbtests'
Torsten Werner [Sun, 6 Mar 2011 21:25:42 +0000 (22:25 +0100)]
Add missing 'dists' directory to method output_filename().
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Sat, 5 Mar 2011 18:55:01 +0000 (19:55 +0100)]
Fully rename bootstrap_bin to scan.
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Sat, 5 Mar 2011 17:55:00 +0000 (17:55 +0000)]
bugfixes
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Sat, 5 Mar 2011 17:26:52 +0000 (18:26 +0100)]
Rewrite most of dak/contents.py.
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Sat, 5 Mar 2011 15:23:25 +0000 (16:23 +0100)]
Update ContentsTestCase to the current formatting.
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Sat, 5 Mar 2011 11:21:55 +0000 (11:21 +0000)]
some bugfixes
Signed-off-by: Torsten Werner <twerner@debian.org>
Torsten Werner [Sat, 5 Mar 2011 11:19:18 +0000 (12:19 +0100)]
Implement class method ContentsWriter.write_all().
Signed-off-by: Torsten Werner <twerner@debian.org>