X-Git-Url: https://git.decadent.org.uk/gitweb/?a=blobdiff_plain;f=TODO;h=389958c4e8adc264d53d0a7bf1293ec3bac21c4b;hb=9540d873fa78598454af57f5f8a4875969ed0439;hp=41e49c61dba4484dfc67814120769a66bd27b0db;hpb=482a176de44e67c36a8a1a911383e960f9a82c32;p=dak.git diff --git a/TODO b/TODO index 41e49c61..389958c4 100644 --- a/TODO +++ b/TODO @@ -1,30 +1,224 @@ TODO ==== +[NB: I use this as a thought record/scribble, not everything on here + makes sense and/or is actually ever going to get done, so IIWY I + wouldn't use it as gospel for the future of katie or as a TODO + list for random hacking.] + ================================================================================ -Actually Urgent ---------------- +Others +------ - o katie's per-suite version checks don't work for stable uploads - since "stable" has been removed from the list of target suites - (doh). + o cindy should remove the src-only override when a binary+source override + exists - o lisa doesn't do the right thing with -2 and -1 uploads, as you can - end up with the .orig.tar.gz not in the pool + o reject on > or < in a version constraint - o lisa can't handle non-existent .changes files +23:07 < aba> elmo: and, how about enhancing rene to spot half-dropped + binaries on one arch (i.e. package used to build A and B, but B is + no longer built on some archs)? - o lisa doesn't handle :q! from rejections well at all + o tabnanny the source - o lisa exits if you check twice (aj) + o drop map-unreleased -More Urgent ------------ + o check email only portions of addresses match too, iff the names + don't, helps with the "James Troup " + vs. "" case. + + o ensure .dsc section/prio match .changes section/prio + + o rhona's kind of crap when asked to remove a lot of files (e.g. 2k + or so). + + o we don't handle the case where an identical orig.tar.gz is + mentioned in the .changes, but not in unchecked; but should we + care? + + o madison could do better sanity checking for -g/-G (e.g. not more + than one suite, etc.) + + o use python2.2-tarfile (once it's in stable?) to check orig.tar.gz + timestamps too. + + o need to decide on whether we're tying for most errors at once.. if + so (probably) then make sure code doesn't assume variables exist and + either way do something about checking error code of check_dsc and + later functions so we skip later checks if they're bailing. + + o the .katie stuff is fundamentally braindamaged, it's not versioned + so there's no way to change the format, yay me. need to fix. + probably by putting a version var as the first thing and checking + that.. auto-upgrade at least from original format would be good. + might also be a good idea to put everything in one big dict after + that? + + o [?, wishlist, distant future] RFC2047-ing should be extended to + all headers of mails sent out. + + o reject sparc64 binaries in a non '*64*' package. + + o katie.py(source_exists): a) we take arguments as parameters that + we could figure out for ourselves (we're part of the Katie class + after all), b) we have this 3rd argument which defaults to "any" + but could in fact be dropped since no one uses it like that. + + o jennifer: doesn't handle bin-only NMUs of stuff still in NEW, + BYHAND or ACCEPTED (but not the pool) - not a big deal, upload can + be retried once the source is in the archive, but still. + + o security global mail overrides should special case buildd stuff so + that buildds get ACCEPTED mails (or maybe amber (?)), that way + upload-security doesn't grow boundlessly. + + o amber should upload sourceful packages first, otherwise with big + packages (e.g. X) and esp. when source is !i386, half the arches + can be uploaded without source, get copied into queue/unaccepted + and promptly rejected. + + o rene's NVIU check doesn't catch cases where source package changed + name, should check binaries too. [debian-devel@l.d.o, 2004-02-03] + + o cnf[melanie::logfile] is misnamed... + + i'd be kinda inclined to go with insisting the .changes file take + the form ---- BEGIN PGP MESSAGE --- -- + BEGIN PGP SIG -- END PGP MESSAGE -- with no lines before or after, + and rejecting .changes that didn't match that + + o rene should check for source packages not building any binaries + + o heidi should have a diff mode that accepts diff output! + + o halle doesn't deal with melanie'd packages, partial replacements + etc. and more. + + o lauren, the tramp, blindly deletes with no check that the delete + failed which it might well given we only look for package/version, + not package/version _in p-u_. duh. + + o melanie should remove obsolete changes when removing from p-u, or + at least warn. or halle should handle it. + + o need a testsuite _badly_ + + o lisa should have an Bitch-Then-Accept option + + o jennifer crashes if run as a user in -n mode when orig.tar.gz is + in queue/new... + + [maybe I should reject debian packages with a non-Debian origin or bugs field] + [agreed; dunno what origin does but non-Debian bugs fields would be bad] + + o rhona should make use of select..except select, temporary tables + etc. rather than looping and calling SQL every time so we can do + suite removal sanely (see potato-removal document) + + o melanie will happily include packages in the Cc list that aren't + being removed... + + o melanie doesn't remove udebs when removing the source they build from + + o check_dsc_against_db's "delete an entry from files while you're + not looking" habit is Evil and Bad. + + o lisa allows you to edit the section and change the component, but + really shouldn't. + + o melanie needs to, when not sending bug close mails, promote Cc: to + To: and send the mail anyways. + + o the lockfile (Archive_Maintenance_In_Progress) should probably be in a conf file + + o madison should cross-check the b.source field and if it's not null + and s.name linked from it != the source given in + -S/--source-and-binary ignore. + + o lauren sucks; she should a) only spam d-i for sourceful + rejections, b) sort stuff so she rejects sourceful stuff first. the + non-sourceful should probably get a form mail, c) automate the + non-sourceful stuff (see b). + + o jennifer should do q-d stuff for faster AA [ryan] + + o split the morgue into source and binary so binaries can be purged first! + + o per-architecture priorities for things like different arch'es + gcc's, silly BSD libftw, palo, etc. + + o use postgres 7.2's built-in stat features to figure out how indices are used etc. + + o neve shouldn't be using location, she should run down suites instead + + o halle needs to know about udebs + + o by default hamstring katie's mail sending so that she won't send + anything until someone edits a script; she's been used far too + much to send spam atm :( + + o $ftpdir/indices isn't created by rose because it's not in katie.conf + + o sanity check depends/recommends/suggests too? in fact for any + empty field? + +[minor] kelly's copychanges, copykatie handling sucks, the per-suite + thing is static for all packages, so work out in advance dummy. + +[madison] # filenames ? +[madison] # maintainer, component, install date (source only?), fingerprint? + + o UrgencyLog stuff should minimize it's bombing out(?) + o Log stuff should open the log file + + o helena should footnote the actual notes, and also * the versions + with notes so we can see new versions since being noted... - o jenna sucks + o helena should have alternative sorting options, including reverse + and without or without differentiaion. - o queued and/or perl on sparc stable sucks + o julia should sync debadmin and ftpmaster (?) + + o Can't read file.: + /org/security.debian.org/queue/accepted/accepted/apache-perl_1.3.9-14.1-1.21.20000309-1_sparc.katie. + You assume that the filenames are relative to accepted/, might want + to doc or fix that. + + o the orig was in NEW, the changes that caused it to be NEW + were pulled out in -2, and we end up with no orig in the archive + :( + + o SecurityQueueBuild doesn't handle the case of foo_3.3woody1 + with a new .orig.tar.gz followed by a foo_3.3potato1 with the same + .orig.tar.gz; jennifer sees it and copes, but the AA code doesn't + and can't really easily know so the potato AA dir is left with no + .orig.tar.gz copy. doh. + + o orig.tar.gz in accepted not handled properly (?) + + o amber doesn't include .orig.tar.gz but it should + + o permissions (paranoia, group write, etc.) configurability and overhaul + + o remember duplicate copyrights in lisaand skip them, per package + + o ove option for lisa byhand proecessing + + o rene could do with overrides + + o db_access.get_location_id should handle the lack of archive_id properly + + o the whole versioncmp thing should be documented + + o lisa doesn't do the right thing with -2 and -1 uploads, as you can + end up with the .orig.tar.gz not in the pool + + o lisa exits if you check twice (aj) + + o lisa doesn't trap signals from fernanda properly + + o queued and/or perl on sparc stable sucks - reimplement it. o aj's bin nmu changes @@ -33,40 +227,29 @@ More Urgent * arch != {any,all} * build-depends wrong (via andrea) * suid - * conficlits + * conflicts * notification/stats to admin daily o trap fernanda exiting o distinguish binary only versus others (neuro) - o change utils.copy to try rename() first - o cache changes parsed from ordering (careful tho: would be caching - .changes from world writable incoming, not holding) + .changes from world writable incoming, not holding) o katie doesn't recognise binonlyNMUs correctly in terms of telling who their source is; source-must-exist does, but the info is not propogated down. - o DONE - - o stable installs are FUBAR - o clean fingerprint - o fix up cron.weekly and/or shania - o Fix BTS vs. katie sync issues by queueing(via BSMTP) BTS mail so that it can be released on deman (e.g. ETRN to exim). - o write our own locking module or in some other way fix things so we - don't require a patch to python since upstream refuse to fix it. - - o maintainers file needs overrides - + o maintainers file needs overrides + [ change override.maintainer to override.maintainer-from + override.maintainer-to and have them reference the maintainers table. Then fix charisma to use them and write some scripting to handle the Santiago situation. ] - o (Depends) elmo: it should match \(\s*(<<|<|<=|=|>=|>|>>)\s*\) fwiw + o Validate Depends (et al.) [it should match \(\s*(<<|<|<=|=|>=|>|>>)\s*\)] o Clean up DONE; archive to tar file every 2 weeks, update tar tvzf INDEX file. @@ -74,21 +257,51 @@ More Urgent unstable and source-ver ~= source-ver in testing; then map unstable -> testing-updates ? - o [Hard] dependency checking (esp. stable) (partially done) + o hooks or configurability for debian specific checks (e.g. check_urgency, auto-building support) - o hooks for debian specific checks (e.g. check_urgency) + o morgue needs auto-cleaning (?) - o Logging [mostly done] + o saffron: two modes, all included, seperate + o saffron: add non-US + o saffron: add ability to control components, architectures, archives, suites + o saffron: add key to expand header - o Abstraction of suite hardcoding (q.v. progengy) +================================================================================ - o morgue needs auto-cleaning (?) +queue/approved +-------------- + + o What to do with multi-suite uploads? Presumably hold in unapproved + and warn? Or what? Can't accept just for unstable or reject just + from stable. + + o Whenever we check for anything in accepted we also need to check in + unapproved. + + o non-sourceful uploads should go straight through if they have + source in accepted or the archive. + + o security uploads on auric should be pre-approved. ================================================================================ Less Urgent ----------- + o change utils.copy to try rename() first + + o [hard, long term] unchecked -> accepted should go into the db, not + a suite, but similar. this would allow katie to get even faster, + make madison more useful, decomplexify specialacceptedautobuild + and generally be more sane. may even be helpful to have e.g. new + in the DB, so that we avoid corner cases like the .orig.tar.gz + disappearing 'cos the package has been entirely removed but was + still on stayofexecution when it entered new. + + o Logging [mostly done] (todo: rhona (hard), .. ?) + + o jennifer: the tar extractor class doesn't need to be redone for each package + o reverse of source-must-exist; i.e. binary-for-source-must-not-exist o REJECT reminders in shania. o fernanda should check for conflicts and warn about them visavis priority [rmurray] @@ -104,10 +317,8 @@ Less Urgent o To incorporate from utils: @ unreject - o katie could vrfy email addresses (?) [errge@] o auto-purge out-of-date stuff from non-free/contrib so that testing and stuff works o doogie's binary -> source index - o Add urgency + installed date to the DB for aj/testing (? still needed ?) o jt's web stuff, matt's changelog stuff (overlap) o [Hard] Need to merge non-non-US and non-US DBs. @@ -120,6 +331,7 @@ Less Urgent o Improve andrea. o Need to optimize all the queries by using EXAMINE and building some INDEXs. + [postgresql 7.2 will help here] o Need to enclose all the setting SQL stuff in transactions (mostly done). o Need to finish alyson (a way to sync katie.conf and the DB) o Need the ability to rebuild all other tables from dists _or_ pools (in the event of disaster) (?) @@ -136,18 +348,23 @@ Less Urgent @ backup of report (?) @ backup of changes.tgz (?) + o --help doesn't work without /etc/katie/katie.conf (or similar) at + least existing. + + o rename andrea (clashes with existing andrea)... + * Harder: + o interrupting of stracing jennifer causes exceptions errors from apt_inst calls + o dependency checking (esp. stable) (partially done) o override checks sucks; it needs to track changes made by the maintainer and pass them onto ftpmaster instead of warning the maintainer. o need to do proper rfc822 escaping of from lines (as opposed to s/\.//g) - o when dinstall is run in install mode but not as a cron job, it - should probably run jenna to avoid madison originated user confusion o Revisit linking of binary->source in install() in katie. o Fix component handling in overrides (aj) o Fix lack of entires in source overrides (aj) - o direport misreports things as section 'devel' + o direport misreports things as section 'devel' (? we don't use direport) o vrfy check of every Maintainer+Changed-By address; valid for 3 months. o binary-all should be done on a per-source, per-architecture package basis to avoid, e.g. the perl-modules problem. @@ -161,41 +378,39 @@ Less Urgent o katie/jenna/rhona/whatever needs to clear out .changes files from p-u when removing stuff superseded by newer versions. [but for now we have halle] - o test sig checking stuff in test/ (stupid thing is not modularized) + o test sig checking stuff in test/ (stupid thing is not modularized due to global abuse) o when encountering suspicous things (e.g. file tainting) do something more drastic * Easy: + o suite mapping and component mapping are parsed per changes file, + they should probably be stored in a dictionary created at startup. + o don't stat/md5sum files you have entries for in the DB, moron + boy (Katie.check_source_blah_blah) o promote changes["changes"] to mandatory in katie.py(dump_vars) after a month or so (or all .katie files contain in the queue contain it). - o Rationalize config stuff.. Dir::Foo not Dir::FooDir [breaks other people??] - Also Dir::Queue::Foo, not Dir::QueueFooDir o melanie should behave better with -a and without -b; see gcc-defaults removal for an example. o Reject on misconfigured kernel-package uploads - o denise abuses sys.stdout == badly o utils.extract_component_from_section: main/utils -> main/utils, main rather than utils, main - o Fix katie to warn if run when not in incoming or p-u - o check for empty debs; check for empty (or small (<1k ?)) .tar.gz's. + o Fix jennier to warn if run when not in incoming or p-u o katie should validate multi-suite uploads; only possible valid one is "stable unstable" o cron.daily* should change umask (aj sucks) o Rene doesn't look at debian-installer but should. o Rene needs to check for binary-less source packages. o Rene could accept a suite argument (?) - o run shania every day (?) [ryan] - o we don't check for .orig.tar.gz's, so it's possible to upload -2 with no .orig.tar.gz. o byhand stuff should send notification o catherine should udpate db; move files, not the other way around [neuro] o melanie should update the stable changelog [joey] o update tagdb.dia - + * Bizzare/uncertain: - - o validate source consistency, i.e. .dsc and .tar.gz as minimum (??) + + o drop rather dubious currval stuff (?) + o rationalize os.path.join() usage o Rene also doesn't seem to warn about missing binary packages (??) - o jenna should not delete things because they don't exist (?) o logging: hostname + pid ? o ANAIS should be done in katie (?) o Add an 'add' ability to melanie (? separate prog maybe) @@ -203,20 +418,13 @@ Less Urgent o Handle the case of 1:1.1 which would overwrite 1.1 (?) o maybe drop -r/--regex in madison, make it the default and implement -e/--exact (a la joey's "elmo") - - * Cleanups & misc: - - o db_access' get_files needs to use exceptions not this None, > 0, < 0 return val BS (?) - o The untouchable flag doesn't stop new packages being added to ``untouchable'' suites - o jenna is too slow - o jenna doesn't handle arch: any -> arch: all transitions - [aj worked around; need to revisit] + o dsc files are not checked for existence/perms (only an issue if + they're in the .dsc, but not the .changes.. possible?) - * Stable: + * Cleanups & misc: - well, there's one other thing that doesn't work so well - you probably should have stable_install ignore Closes: - or it might be unique to the case of stable unstable uploads + o db_access' get_files needs to use exceptions not this None, > 0, < 0 return val BS (?) + o The untouchable flag doesn't stop new packages being added to ``untouchable'' suites ================================================================================ @@ -225,4 +433,54 @@ Packaging o Fix stuff to look in sensible places for libs and config file in debian package (?) +================================================================================ + + --help manpage +----------------------------------------------------------------------------- +alyson X +amber X +andrea X +ashley X +catherine X X +charisma X X +cindy X X +claire X +denise X +fernanda X +halle X +heidi X X +helena X +jenna X +jennifer X +jeri X +julia X X +kelly X X +lisa X X +madison X X +melanie X X +natalie X X +neve X +rene X +rose X +rhona X X +saffron X +shania X +tea X +ziyi X + + +================================================================================ + +Random useful-at-some-point SQL +------------------------------- + +UPDATE files SET last_used = '1980-01-01' + FROM binaries WHERE binaries.architecture = + AND binaries.file = files.id; + +DELETE FROM bin_associations + WHERE EXISTS (SELECT id FROM binaries + WHERE architecture = + AND id = bin_associations.bin); + ================================================================================