4 [NB: I use this as a thought record/scribble, not everything on here
5 makes sense and/or is actually ever going to get done, so IIWY I
6 wouldn't use it as gospel for the future of katie or as a TODO
7 list for random hacking.]
9 ================================================================================
14 o reject on > or < in a version constraint
16 23:07 < aba> elmo: and, how about enhancing rene to spot half-dropped
17 binaries on one arch (i.e. package used to build A and B, but B is
18 no longer built on some archs)?
24 o check email only portions of addresses match too, iff the names
25 don't, helps with the "James Troup <james@nocrew.org>"
26 vs. "<james@nocrew.org>" case.
28 o ensure .dsc section/prio match .changes section/prio
30 o rhona's kind of crap when asked to remove a lot of files (e.g. 2k
33 o we don't handle the case where an identical orig.tar.gz is
34 mentioned in the .changes, but not in unchecked; but should we
37 o madison could do better sanity checking for -g/-G (e.g. not more
40 o use python2.2-tarfile (once it's in stable?) to check orig.tar.gz
43 o need to decide on whether we're tying for most errors at once.. if
44 so (probably) then make sure code doesn't assume variables exist and
45 either way do something about checking error code of check_dsc and
46 later functions so we skip later checks if they're bailing.
48 o the .katie stuff is fundamentally braindamaged, it's not versioned
49 so there's no way to change the format, yay me. need to fix.
50 probably by putting a version var as the first thing and checking
51 that.. auto-upgrade at least from original format would be good.
52 might also be a good idea to put everything in one big dict after
55 o [?, wishlist, distant future] RFC2047-ing should be extended to
56 all headers of mails sent out.
58 o reject sparc64 binaries in a non '*64*' package.
60 o katie.py(source_exists): a) we take arguments as parameters that
61 we could figure out for ourselves (we're part of the Katie class
62 after all), b) we have this 3rd argument which defaults to "any"
63 but could in fact be dropped since no one uses it like that.
65 o jennifer: doesn't handle bin-only NMUs of stuff still in NEW,
66 BYHAND or ACCEPTED (but not the pool) - not a big deal, upload can
67 be retried once the source is in the archive, but still.
69 o security global mail overrides should special case buildd stuff so
70 that buildds get ACCEPTED mails (or maybe amber (?)), that way
71 upload-security doesn't grow boundlessly.
73 o amber should upload sourceful packages first, otherwise with big
74 packages (e.g. X) and esp. when source is !i386, half the arches
75 can be uploaded without source, get copied into queue/unaccepted
76 and promptly rejected.
78 o rene's NVIU check doesn't catch cases where source package changed
79 name, should check binaries too. [debian-devel@l.d.o, 2004-02-03]
81 o cnf[melanie::logfile] is misnamed...
83 <aj> i'd be kinda inclined to go with insisting the .changes file take
84 the form ---- BEGIN PGP MESSAGE --- <non -- BEGIN/END lines> --
85 BEGIN PGP SIG -- END PGP MESSAGE -- with no lines before or after,
86 and rejecting .changes that didn't match that
88 o rene should check for source packages not building any binaries
90 o heidi should have a diff mode that accepts diff output!
92 o halle doesn't deal with melanie'd packages, partial replacements
95 o lauren, the tramp, blindly deletes with no check that the delete
96 failed which it might well given we only look for package/version,
97 not package/version _in p-u_. duh.
99 o melanie should remove obsolete changes when removing from p-u, or
100 at least warn. or halle should handle it.
102 o need a testsuite _badly_
104 o lisa should have an Bitch-Then-Accept option
106 o jennifer crashes if run as a user in -n mode when orig.tar.gz is
109 <elmo_home> [<random>maybe I should reject debian packages with a non-Debian origin or bugs field</>]
110 <Kamion> [<random>agreed; dunno what origin does but non-Debian bugs fields would be bad]
112 o rhona should make use of select..except select, temporary tables
113 etc. rather than looping and calling SQL every time so we can do
114 suite removal sanely (see potato-removal document)
116 o melanie will happily include packages in the Cc list that aren't
119 o melanie doesn't remove udebs when removing the source they build from
121 o check_dsc_against_db's "delete an entry from files while you're
122 not looking" habit is Evil and Bad.
124 o lisa allows you to edit the section and change the component, but
127 o melanie needs to, when not sending bug close mails, promote Cc: to
128 To: and send the mail anyways.
130 o the lockfile (Archive_Maintenance_In_Progress) should probably be in a conf file
132 o madison should cross-check the b.source field and if it's not null
133 and s.name linked from it != the source given in
134 -S/--source-and-binary ignore.
136 o lauren sucks; she should a) only spam d-i for sourceful
137 rejections, b) sort stuff so she rejects sourceful stuff first. the
138 non-sourceful should probably get a form mail, c) automate the
139 non-sourceful stuff (see b).
141 o jennifer should do q-d stuff for faster AA [ryan]
143 o split the morgue into source and binary so binaries can be purged first!
145 o per-architecture priorities for things like different arch'es
146 gcc's, silly BSD libftw, palo, etc.
148 o use postgres 7.2's built-in stat features to figure out how indices are used etc.
150 o neve shouldn't be using location, she should run down suites instead
152 o halle needs to know about udebs
154 o by default hamstring katie's mail sending so that she won't send
155 anything until someone edits a script; she's been used far too
156 much to send spam atm :(
158 o $ftpdir/indices isn't created by rose because it's not in katie.conf
160 o sanity check depends/recommends/suggests too? in fact for any
163 [minor] kelly's copychanges, copykatie handling sucks, the per-suite
164 thing is static for all packages, so work out in advance dummy.
166 [madison] # filenames ?
167 [madison] # maintainer, component, install date (source only?), fingerprint?
169 o UrgencyLog stuff should minimize it's bombing out(?)
170 o Log stuff should open the log file
172 o helena should footnote the actual notes, and also * the versions
173 with notes so we can see new versions since being noted...
175 o helena should have alternative sorting options, including reverse
176 and without or without differentiaion.
178 o julia should sync debadmin and ftpmaster (?)
180 o <drow> Can't read file.:
181 /org/security.debian.org/queue/accepted/accepted/apache-perl_1.3.9-14.1-1.21.20000309-1_sparc.katie.
182 You assume that the filenames are relative to accepted/, might want
185 o <neuro> the orig was in NEW, the changes that caused it to be NEW
186 were pulled out in -2, and we end up with no orig in the archive
189 o SecurityAcceptedAutoBuild doesn't handle the case of foo_3.3woody1
190 with a new .orig.tar.gz followed by a foo_3.3potato1 with the same
191 .orig.tar.gz; jennifer sees it and copes, but the AA code doesn't
192 and can't really easily know so the potato AA dir is left with no
193 .orig.tar.gz copy. doh.
195 o orig.tar.gz in accepted not handled properly (?)
197 o amber doesn't include .orig.tar.gz but it should
199 o permissions (paranoia, group write, etc.) configurability and overhaul
201 o remember duplicate copyrights in lisaand skip them, per package
203 o <M>ove option for lisa byhand proecessing
205 o rene could do with overrides
207 o db_access.get_location_id should handle the lack of archive_id properly
209 o the whole versioncmp thing should be documented
211 o lisa doesn't do the right thing with -2 and -1 uploads, as you can
212 end up with the .orig.tar.gz not in the pool
214 o lisa exits if you check twice (aj)
216 o lisa doesn't trap signals from fernanda properly
218 o queued and/or perl on sparc stable sucks - reimplement it.
220 o aj's bin nmu changes
223 * priority >> optional
225 * build-depends wrong (via andrea)
228 * notification/stats to admin daily
229 o trap fernanda exiting
230 o distinguish binary only versus others (neuro)
232 o cache changes parsed from ordering (careful tho: would be caching
233 .changes from world writable incoming, not holding)
235 o katie doesn't recognise binonlyNMUs correctly in terms of telling
236 who their source is; source-must-exist does, but the info is not
239 o Fix BTS vs. katie sync issues by queueing(via BSMTP) BTS mail so
240 that it can be released on deman (e.g. ETRN to exim).
242 o maintainers file needs overrides
244 [ change override.maintainer to override.maintainer-from +
245 override.maintainer-to and have them reference the maintainers
246 table. Then fix charisma to use them and write some scripting
247 to handle the Santiago situation. ]
249 o Validate Depends (et al.) [it should match \(\s*(<<|<|<=|=|>=|>|>>)\s*<VERSIONREGEXP>\)]
251 o Clean up DONE; archive to tar file every 2 weeks, update tar tvzf INDEX file.
253 o testing-updates suite: if binary-only and version << version in
254 unstable and source-ver ~= source-ver in testing; then map
255 unstable -> testing-updates ?
257 o hooks or configurability for debian specific checks (e.g. check_urgency, auto-building support)
259 o morgue needs auto-cleaning (?)
261 o saffron: two modes, all included, seperate
262 o saffron: add non-US
263 o saffron: add ability to control components, architectures, archives, suites
264 o saffron: add key to expand header
266 ================================================================================
271 o What to do with multi-suite uploads? Presumably hold in unapproved
272 and warn? Or what? Can't accept just for unstable or reject just
275 o Whenever we check for anything in accepted we also need to check in
278 o non-sourceful uploads should go straight through if they have
279 source in accepted or the archive.
281 o security uploads on auric should be pre-approved.
283 ================================================================================
288 o change utils.copy to try rename() first
290 o [hard, long term] unchecked -> accepted should go into the db, not
291 a suite, but similar. this would allow katie to get even faster,
292 make madison more useful, decomplexify specialacceptedautobuild
293 and generally be more sane. may even be helpful to have e.g. new
294 in the DB, so that we avoid corner cases like the .orig.tar.gz
295 disappearing 'cos the package has been entirely removed but was
296 still on stayofexecution when it entered new.
298 o Logging [mostly done] (todo: rhona (hard), .. ?)
300 o jennifer: the tar extractor class doesn't need to be redone for each package
302 o reverse of source-must-exist; i.e. binary-for-source-must-not-exist
303 o REJECT reminders in shania.
304 o fernanda should check for conflicts and warn about them visavis priority [rmurray]
305 o store a list of removed/files versions; also compare against them.
306 [but be careful about scalability]
308 o fernanda: print_copyright should be a lot more intelligent
309 @ handle copyright.gz
310 @ handle copyright.ja and copyright
311 @ handle (detect at least) symlinks to another package's doc directory
312 @ handle and/or fall back on source files (?)
314 o To incorporate from utils:
317 o auto-purge out-of-date stuff from non-free/contrib so that testing and stuff works
318 o doogie's binary -> source index
319 o jt's web stuff, matt's changelog stuff (overlap)
321 o [Hard] Need to merge non-non-US and non-US DBs.
323 o experimental needs to auto clean (relative to unstable) [partial: rene warns about this]
325 o Do a checkpc(1)-a-like which sanitizes a config files.
326 o fix parse_changes()/build_file_list() to sanity check filenames
327 o saftey check and/or rename debs so they match what they should be
330 o Need to optimize all the queries by using EXAMINE and building some INDEXs.
331 [postgresql 7.2 will help here]
332 o Need to enclose all the setting SQL stuff in transactions (mostly done).
333 o Need to finish alyson (a way to sync katie.conf and the DB)
334 o Need the ability to rebuild all other tables from dists _or_ pools (in the event of disaster) (?)
335 o Make the --help and --version options do stuff for all scripts
337 o charisma can't handle whitespace-only lines (for the moment, this is feature)
339 o generic way of saying isabinary and isadsc. (?)
341 o s/distribution/suite/g
344 @ weekly postins to d-c (?)
345 @ backup of report (?)
346 @ backup of changes.tgz (?)
348 o --help doesn't work without /etc/katie/katie.conf (or similar) at
351 o rename andrea (clashes with existing andrea)...
355 o interrupting of stracing jennifer causes exceptions errors from apt_inst calls
356 o dependency checking (esp. stable) (partially done)
357 o override checks sucks; it needs to track changes made by the
358 maintainer and pass them onto ftpmaster instead of warning the
360 o need to do proper rfc822 escaping of from lines (as opposed to s/\.//g)
361 o Revisit linking of binary->source in install() in katie.
362 o Fix component handling in overrides (aj)
363 o Fix lack of entires in source overrides (aj)
364 o direport misreports things as section 'devel' (? we don't use direport)
365 o vrfy check of every Maintainer+Changed-By address; valid for 3 months.
366 o binary-all should be done on a per-source, per-architecture package
367 basis to avoid, e.g. the perl-modules problem.
368 o a source-missing-diff check: if the version has a - in it, and it
369 is sourceful, it needs orig and diff, e.g. if someone uploads
370 esound_0.2.22-6, and it is sourceful, and there is no diff ->
371 REJECT (version has a dash, therefore not debian native.)
372 o check linking of .tar.gz's to .dsc's.. see proftpd 1.2.1 as an example
373 o archive needs md5sum'ed regularly, but takes too long to do all
374 in one go; make progressive or weekly.
375 o katie/jenna/rhona/whatever needs to clear out .changes
376 files from p-u when removing stuff superseded by newer versions.
377 [but for now we have halle]
378 o test sig checking stuff in test/ (stupid thing is not modularized due to global abuse)
379 o when encountering suspicous things (e.g. file tainting) do something more drastic
383 o suite mapping and component mapping are parsed per changes file,
384 they should probably be stored in a dictionary created at startup.
385 o don't stat/md5sum files you have entries for in the DB, moron
386 boy (Katie.check_source_blah_blah)
387 o promote changes["changes"] to mandatory in katie.py(dump_vars)
388 after a month or so (or all .katie files contain in the queue
390 o melanie should behave better with -a and without -b; see
391 gcc-defaults removal for an example.
392 o Reject on misconfigured kernel-package uploads
393 o utils.extract_component_from_section: main/utils -> main/utils, main rather than utils, main
394 o Fix jennier to warn if run when not in incoming or p-u
395 o katie should validate multi-suite uploads; only possible valid one
397 o cron.daily* should change umask (aj sucks)
398 o Rene doesn't look at debian-installer but should.
399 o Rene needs to check for binary-less source packages.
400 o Rene could accept a suite argument (?)
401 o byhand stuff should send notification
402 o catherine should udpate db; move files, not the other way around [neuro]
403 o melanie should update the stable changelog [joey]
408 o drop rather dubious currval stuff (?)
409 o rationalize os.path.join() usage
410 o Rene also doesn't seem to warn about missing binary packages (??)
411 o logging: hostname + pid ?
412 o ANAIS should be done in katie (?)
413 o Add an 'add' ability to melanie (? separate prog maybe)
414 o Replicate old dinstall report stuff (? needed ?)
415 o Handle the case of 1:1.1 which would overwrite 1.1 (?)
416 o maybe drop -r/--regex in madison, make it the default and
417 implement -e/--exact (a la joey's "elmo")
418 o dsc files are not checked for existence/perms (only an issue if
419 they're in the .dsc, but not the .changes.. possible?)
423 o db_access' get_files needs to use exceptions not this None, > 0, < 0 return val BS (?)
424 o The untouchable flag doesn't stop new packages being added to ``untouchable'' suites
426 ================================================================================
431 o Fix stuff to look in sensible places for libs and config file in debian package (?)
433 ================================================================================
436 -----------------------------------------------------------------------------
469 ================================================================================
471 Random useful-at-some-point SQL
472 -------------------------------
474 UPDATE files SET last_used = '1980-01-01'
475 FROM binaries WHERE binaries.architecture = <x>
476 AND binaries.file = files.id;
478 DELETE FROM bin_associations
479 WHERE EXISTS (SELECT id FROM binaries
480 WHERE architecture = <x>
481 AND id = bin_associations.bin);
483 ================================================================================