4 [NB: I use this as a thought record/scribble, not everything on here
5 makes sense and/or is actually ever going to get done, so IIWY I
6 wouldn't use it as gospel for the future of dak or as a TODO
7 list for random hacking.]
9 ================================================================================
14 o 'dak check-overrides' should remove the src-only override when a
15 binary+source override exists
17 o reject on > or < in a version constraint
19 23:07 < aba> elmo: and, how about enhancing 'dak cruft-report' to spot half-dropped
20 binaries on one arch (i.e. package used to build A and B, but B is
21 no longer built on some archs)?
27 o check email only portions of addresses match too, iff the names
28 don't, helps with the "James Troup <james@nocrew.org>"
29 vs. "<james@nocrew.org>" case.
31 o ensure .dsc section/prio match .changes section/prio
33 o 'dak clean-suites' performance is kind of crap when asked to
34 remove a lot of files (e.g. 2k or so).
36 o we don't handle the case where an identical orig.tar.gz is
37 mentioned in the .changes, but not in unchecked; but should we
40 o 'dak ls' could do better sanity checking for -g/-G (e.g. not more
43 o use python2.2-tarfile (once it's in stable?) to check orig.tar.gz
46 o need to decide on whether we're tying for most errors at once.. if
47 so (probably) then make sure code doesn't assume variables exist and
48 either way do something about checking error code of check_dsc and
49 later functions so we skip later checks if they're bailing.
51 o the .dak stuff is fundamentally braindamaged, it's not versioned
52 so there's no way to change the format, yay me. need to fix.
53 probably by putting a version var as the first thing and checking
54 that.. auto-upgrade at least from original format would be good.
55 might also be a good idea to put everything in one big dict after
58 o [?, wishlist, distant future] RFC2047-ing should be extended to
59 all headers of mails sent out.
61 o reject sparc64 binaries in a non '*64*' package.
63 o queue.py(source_exists): a) we take arguments as parameters that
64 we could figure out for ourselves (we're part of the Upload class
65 after all), b) we have this 3rd argument which defaults to "any"
66 but could in fact be dropped since no one uses it like that.
68 o 'dak process-unchecked': doesn't handle bin-only NMUs of stuff
69 still in NEW, BYHAND or ACCEPTED (but not the pool) - not a big
70 deal, upload can be retried once the source is in the archive, but
73 o security global mail overrides should special case buildd stuff so
74 that buildds get ACCEPTED mails (or maybe 'dak security-install' (?)), that way
75 upload-security doesn't grow boundlessly.
77 o 'dak security-install' should upload sourceful packages first,
78 otherwise with big packages (e.g. X) and esp. when source is !i386,
79 half the arches can be uploaded without source, get copied into
80 queue/unaccepted and promptly rejected.
82 o 'dak cruft-report's NVIU check doesn't catch cases where source
83 package changed name, should check binaries
84 too. [debian-devel@l.d.o, 2004-02-03]
86 o cnf[Rm::logfile] is misnamed...
88 <aj> i'd be kinda inclined to go with insisting the .changes file take
89 the form ---- BEGIN PGP MESSAGE --- <non -- BEGIN/END lines> --
90 BEGIN PGP SIG -- END PGP MESSAGE -- with no lines before or after,
91 and rejecting .changes that didn't match that
93 o 'dak cruft-report' should check for source packages not building any binaries
95 o 'dak control-suite' should have a diff mode that accepts diff output!
97 o 'dak clean-proposed-updates' doesn't deal with 'dak rm'-d
98 packages, partial replacements etc. and more.
100 o 'dak reject-proposed-updates' blindly deletes with no check that
101 the delete failed which it might well given we only look for
102 package/version, not package/version _in p-u_. duh.
104 o 'dak rm' should remove obsolete changes when removing from p-u, or
105 at least warn. or 'dak reject-proposed-updates' should handle it.
107 o need a testsuite _badly_
109 o 'dak process-unchecked' crashes if run as a user in -n mode when
110 orig.tar.gz is in queue/new...
112 <elmo_home> [<random>maybe I should reject debian packages with a non-Debian origin or bugs field</>]
113 <Kamion> [<random>agreed; dunno what origin does but non-Debian bugs fields would be bad]
115 o 'dak clean-suites' should make use of select..except select, temporary tables
116 etc. rather than looping and calling SQL every time so we can do
117 suite removal sanely (see potato-removal document)
119 o 'dak rm' will happily include packages in the Cc list that aren't
122 o 'dak rm' doesn't remove udebs when removing the source they build from
124 o check_dsc_against_db's "delete an entry from files while you're
125 not looking" habit is Evil and Bad.
127 o 'dak process-new' allows you to edit the section and change the
128 component, but really shouldn't.
130 o 'dak rm' needs to, when not sending bug close mails, promote Cc: to
131 To: and send the mail anyways.
133 o the lockfile (Archive_Maintenance_In_Progress) should probably be in a conf file
135 o 'dak ls' should cross-check the b.source field and if it's not
136 null and s.name linked from it != the source given in
137 -S/--source-and-binary ignore.
139 o 'dak reject-proposed-updates' sucks; it should a) only spam d-i
140 for sourceful rejections, b) sort stuff so it rejects sourceful
141 stuff first. the non-sourceful should probably get a form mail, c)
142 automate the non-sourceful stuff (see b).
144 o 'dak process-unchecked' should do q-d stuff for faster AA [ryan]
146 o split the morgue into source and binary so binaries can be purged first!
148 o per-architecture priorities for things like different arch'es
149 gcc's, silly BSD libftw, palo, etc.
151 o use postgres 7.2's built-in stat features to figure out how indices are used etc.
153 o 'dak init-archive' shouldn't be using location, it should run down suites instead
155 o 'dak clean-proposed-updates' needs to know about udebs
157 o by default hamstring dak's mail sending so that it won't send
158 anything until someone edits a script; it's been used far too
159 much to send spam atm :(
161 o $ftpdir/indices isn't created by 'dak init-dir' because it's not in dak.conf
163 o sanity check depends/recommends/suggests too? in fact for any
166 [minor] 'dak process-accepted's copychanges, copydotdak handling
167 sucks, the per-suite thing is static for all packages, so work out
170 [dak ls] # filenames ?
171 [dak ls] # maintainer, component, install date (source only?), fingerprint?
173 o UrgencyLog stuff should minimize it's bombing out(?)
174 o Log stuff should open the log file
176 o 'dak queue-report' should footnote the actual notes, and also *
177 the versions with notes so we can see new versions since being
180 o 'dak queue-report' should have alternative sorting options, including reverse
181 and without or without differentiaion.
183 o 'dak import-users-from-passwd' should sync debadmin and ftpmaster (?)
185 o <drow> Can't read file.:
186 /org/security.debian.org/queue/accepted/accepted/apache-perl_1.3.9-14.1-1.21.20000309-1_sparc.dak.
187 You assume that the filenames are relative to accepted/, might want
190 o <neuro> the orig was in NEW, the changes that caused it to be NEW
191 were pulled out in -2, and we end up with no orig in the archive
194 o SecurityQueueBuild doesn't handle the case of foo_3.3woody1 with a
195 new .orig.tar.gz followed by a foo_3.3potato1 with the same
196 .orig.tar.gz; 'dak process-unchecked' sees it and copes, but the AA
197 code doesn't and can't really easily know so the potato AA dir is
198 left with no .orig.tar.gz copy. doh.
200 o orig.tar.gz in accepted not handled properly (?)
202 o 'dak security-install' doesn't include .orig.tar.gz but it should
204 o permissions (paranoia, group write, etc.) configurability and overhaul
206 o remember duplicate copyrights in 'dak process-new' and skip them, per package
208 o <M>ove option for 'dak process-new' byhand proecessing
210 o 'dak cruft-report' could do with overrides
212 o database.get_location_id should handle the lack of archive_id properly
214 o the whole versioncmp thing should be documented
216 o 'dak process-new' doesn't do the right thing with -2 and -1 uploads, as you can
217 end up with the .orig.tar.gz not in the pool
219 o 'dak process-new' exits if you check twice (aj)
221 o 'dak process-new' doesn't trap signals from 'dak examine-package' properly
223 o queued and/or perl on sparc stable sucks - reimplement it.
225 o aj's bin nmu changes
228 * priority >> optional
230 * build-depends wrong (via 'dak compare-suites')
233 * notification/stats to admin daily
234 o trap 'dak examine-package' exiting
235 o distinguish binary only versus others (neuro)
237 o cache changes parsed from ordering (careful tho: would be caching
238 .changes from world writable incoming, not holding)
240 o dak doesn't recognise binonlyNMUs correctly in terms of telling
241 who their source is; source-must-exist does, but the info is not
244 o Fix BTS vs. dak sync issues by queueing(via BSMTP) BTS mail so
245 that it can be released on deman (e.g. ETRN to exim).
247 o maintainers file needs overrides
249 [ change override.maintainer to override.maintainer-from +
250 override.maintainer-to and have them reference the maintainers
251 table. Then fix 'dak make-maintainers' to use them and write some scripting
252 to handle the Santiago situation. ]
254 o Validate Depends (et al.) [it should match \(\s*(<<|<|<=|=|>=|>|>>)\s*<VERSIONREGEXP>\)]
256 o Clean up DONE; archive to tar file every 2 weeks, update tar tvzf INDEX file.
258 o testing-updates suite: if binary-only and version << version in
259 unstable and source-ver ~= source-ver in testing; then map
260 unstable -> testing-updates ?
262 o hooks or configurability for debian specific checks (e.g. check_urgency, auto-building support)
264 o morgue needs auto-cleaning (?)
266 o dak stats: two modes, all included, seperate
267 o dak stats: add non-US
268 o dak stats: add ability to control components, architectures, archives, suites
269 o dak stats: add key to expand header
271 ================================================================================
276 o What to do with multi-suite uploads? Presumably hold in unapproved
277 and warn? Or what? Can't accept just for unstable or reject just
280 o Whenever we check for anything in accepted we also need to check in
283 o non-sourceful uploads should go straight through if they have
284 source in accepted or the archive.
286 o security uploads on auric should be pre-approved.
288 ================================================================================
293 o change utils.copy to try rename() first
295 o [hard, long term] unchecked -> accepted should go into the db, not
296 a suite, but similar. this would allow dak to get even faster,
297 make 'dak ls' more useful, decomplexify specialacceptedautobuild
298 and generally be more sane. may even be helpful to have e.g. new
299 in the DB, so that we avoid corner cases like the .orig.tar.gz
300 disappearing 'cos the package has been entirely removed but was
301 still on stayofexecution when it entered new.
303 o Logging [mostly done] (todo: 'dak clean-suites' (hard), .. ?)
305 o 'dak process-unchecked': the tar extractor class doesn't need to be redone for each package
307 o reverse of source-must-exist; i.e. binary-for-source-must-not-exist
308 o REJECT reminders in 'dak clean-queues'.
309 o 'dak examine-package' should check for conflicts and warn about them visavis priority [rmurray]
310 o store a list of removed/files versions; also compare against them.
311 [but be careful about scalability]
313 o dak examine-package: print_copyright should be a lot more intelligent
314 @ handle copyright.gz
315 @ handle copyright.ja and copyright
316 @ handle (detect at least) symlinks to another package's doc directory
317 @ handle and/or fall back on source files (?)
319 o To incorporate from utils:
322 o auto-purge out-of-date stuff from non-free/contrib so that testing and stuff works
323 o doogie's binary -> source index
324 o jt's web stuff, matt's changelog stuff (overlap)
326 o [Hard] Need to merge non-non-US and non-US DBs.
328 o experimental needs to auto clean (relative to unstable) [partial:
329 'dak cruft-report' warns about this]
331 o Do a checkpc(1)-a-like which sanitizes a config files.
332 o fix parse_changes()/build_file_list() to sanity check filenames
333 o saftey check and/or rename debs so they match what they should be
335 o Improve 'dak compare-suites'.
336 o Need to optimize all the queries by using EXAMINE and building some INDEXs.
337 [postgresql 7.2 will help here]
338 o Need to enclose all the setting SQL stuff in transactions (mostly done).
339 o Need to finish 'dak init-db' (a way to sync dak.conf and the DB)
340 o Need the ability to rebuild all other tables from dists _or_ pools (in the event of disaster) (?)
341 o Make the --help and --version options do stuff for all scripts
343 o 'dak make-maintainers' can't handle whitespace-only lines (for the moment, this is feature)
345 o generic way of saying isabinary and isadsc. (?)
347 o s/distribution/suite/g
350 @ weekly postins to d-c (?)
351 @ backup of report (?)
352 @ backup of changes.tgz (?)
354 o --help doesn't work without /etc/dak/dak.conf (or similar) at
357 o rename 'dak compare-suites' (clashes with existing 'dak compare-suites')...
361 o interrupting of stracing 'dak process-unchecked' causes exceptions errors from apt_inst calls
362 o dependency checking (esp. stable) (partially done)
363 o override checks sucks; it needs to track changes made by the
364 maintainer and pass them onto ftpmaster instead of warning the
366 o need to do proper rfc822 escaping of from lines (as opposed to s/\.//g)
367 o Revisit linking of binary->source in install() in dak.
368 o Fix component handling in overrides (aj)
369 o Fix lack of entires in source overrides (aj)
370 o direport misreports things as section 'devel' (? we don't use direport)
371 o vrfy check of every Maintainer+Changed-By address; valid for 3 months.
372 o binary-all should be done on a per-source, per-architecture package
373 basis to avoid, e.g. the perl-modules problem.
374 o a source-missing-diff check: if the version has a - in it, and it
375 is sourceful, it needs orig and diff, e.g. if someone uploads
376 esound_0.2.22-6, and it is sourceful, and there is no diff ->
377 REJECT (version has a dash, therefore not debian native.)
378 o check linking of .tar.gz's to .dsc's.. see proftpd 1.2.1 as an example
379 o archive needs md5sum'ed regularly, but takes too long to do all
380 in one go; make progressive or weekly.
381 o something needs to clear out .changes files from p-u when
382 removing stuff superseded by newer versions. [but for now we have
383 'dak clean-proposed-updates']
384 o test sig checking stuff in test/ (stupid thing is not modularized due to global abuse)
385 o when encountering suspicous things (e.g. file tainting) do something more drastic
389 o suite mapping and component mapping are parsed per changes file,
390 they should probably be stored in a dictionary created at startup.
391 o don't stat/md5sum files you have entries for in the DB, moron
392 boy (Dak.check_source_blah_blah)
393 o promote changes["changes"] to mandatory in dak.py(dump_vars)
394 after a month or so (or all .dak files contain in the queue
396 o 'dak rm' should behave better with -a and without -b; see
397 gcc-defaults removal for an example.
398 o Reject on misconfigured kernel-package uploads
399 o utils.extract_component_from_section: main/utils -> main/utils, main rather than utils, main
400 o Fix 'dak process-unchecked' to warn if run when not in incoming or p-u
401 o dak should validate multi-suite uploads; only possible valid one
403 o cron.daily* should change umask (aj sucks)
404 o 'dak cruft-report' doesn't look at debian-installer but should.
405 o 'dak cruft-report' needs to check for binary-less source packages.
406 o 'dak cruft-report' could accept a suite argument (?)
407 o byhand stuff should send notification
408 o 'dak poolize' should udpate db; move files, not the other way around [neuro]
409 o 'dak rm' should update the stable changelog [joey]
414 o drop rather dubious currval stuff (?)
415 o rationalize os.path.join() usage
416 o 'dak cruft-report' also doesn't seem to warn about missing binary packages (??)
417 o logging: hostname + pid ?
418 o ANAIS should be done in dak (?)
419 o Add an 'add' ability to 'dak rm' (? separate prog maybe)
420 o Replicate old dinstall report stuff (? needed ?)
421 o Handle the case of 1:1.1 which would overwrite 1.1 (?)
422 o maybe drop -r/--regex in 'dak ls', make it the default and
423 implement -e/--exact (a la joey's "elmo")
424 o dsc files are not checked for existence/perms (only an issue if
425 they're in the .dsc, but not the .changes.. possible?)
429 o db_access' get_files needs to use exceptions not this None, > 0, < 0 return val BS (?)
430 o The untouchable flag doesn't stop new packages being added to ``untouchable'' suites
432 ================================================================================
437 o Fix stuff to look in sensible places for libs and config file in debian package (?)
439 ================================================================================
442 -----------------------------------------------------------------------------
445 check-proposed-updates X
446 clean-proposed-updates X
450 control-overrides X X
457 import-users-from-passwd X X
463 make-suite-file-list X
475 ================================================================================
477 Random useful-at-some-point SQL
478 -------------------------------
480 UPDATE files SET last_used = '1980-01-01'
481 FROM binaries WHERE binaries.architecture = <x>
482 AND binaries.file = files.id;
484 DELETE FROM bin_associations
485 WHERE EXISTS (SELECT id FROM binaries
486 WHERE architecture = <x>
487 AND id = bin_associations.bin);
489 ================================================================================