TODO
====
+[NB: I use this as a thought record/scribble, not everything on here
+ makes sense and/or is actually ever going to get done, so IIWY I
+ wouldn't use it as gospel for the future of katie or as a TODO
+ list for random hacking.]
+
================================================================================
-Actually Urgent
----------------
+Others
+------
- o lisa doesn't do the right thing with -2 and -1 uploads, as you can
- end up with the .orig.tar.gz not in the pool
+ o cindy should remove the src-only override when a binary+source override
+ exists
- o lisa exits if you check twice (aj)
+ o reject on > or < in a version constraint
- o lisa doesn't trap signals from fernanda properly
+23:07 < aba> elmo: and, how about enhancing rene to spot half-dropped
+ binaries on one arch (i.e. package used to build A and B, but B is
+ no longer built on some archs)?
-More Urgent
------------
+ o tabnanny the source
+
+ o drop map-unreleased
+
+ o check email only portions of addresses match too, iff the names
+ don't, helps with the "James Troup <james@nocrew.org>"
+ vs. "<james@nocrew.org>" case.
+
+ o ensure .dsc section/prio match .changes section/prio
+
+ o rhona's kind of crap when asked to remove a lot of files (e.g. 2k
+ or so).
+
+ o we don't handle the case where an identical orig.tar.gz is
+ mentioned in the .changes, but not in unchecked; but should we
+ care?
+
+ o madison could do better sanity checking for -g/-G (e.g. not more
+ than one suite, etc.)
+
+ o use python2.2-tarfile (once it's in stable?) to check orig.tar.gz
+ timestamps too.
+
+ o need to decide on whether we're tying for most errors at once.. if
+ so (probably) then make sure code doesn't assume variables exist and
+ either way do something about checking error code of check_dsc and
+ later functions so we skip later checks if they're bailing.
+
+ o the .katie stuff is fundamentally braindamaged, it's not versioned
+ so there's no way to change the format, yay me. need to fix.
+ probably by putting a version var as the first thing and checking
+ that.. auto-upgrade at least from original format would be good.
+ might also be a good idea to put everything in one big dict after
+ that?
+
+ o [?, wishlist, distant future] RFC2047-ing should be extended to
+ all headers of mails sent out.
+
+ o reject sparc64 binaries in a non '*64*' package.
+
+ o katie.py(source_exists): a) we take arguments as parameters that
+ we could figure out for ourselves (we're part of the Katie class
+ after all), b) we have this 3rd argument which defaults to "any"
+ but could in fact be dropped since no one uses it like that.
+
+ o jennifer: doesn't handle bin-only NMUs of stuff still in NEW,
+ BYHAND or ACCEPTED (but not the pool) - not a big deal, upload can
+ be retried once the source is in the archive, but still.
+
+ o security global mail overrides should special case buildd stuff so
+ that buildds get ACCEPTED mails (or maybe amber (?)), that way
+ upload-security doesn't grow boundlessly.
+
+ o amber should upload sourceful packages first, otherwise with big
+ packages (e.g. X) and esp. when source is !i386, half the arches
+ can be uploaded without source, get copied into queue/unaccepted
+ and promptly rejected.
+
+ o rene's NVIU check doesn't catch cases where source package changed
+ name, should check binaries too. [debian-devel@l.d.o, 2004-02-03]
+
+ o cnf[melanie::logfile] is misnamed...
+
+<aj> i'd be kinda inclined to go with insisting the .changes file take
+ the form ---- BEGIN PGP MESSAGE --- <non -- BEGIN/END lines> --
+ BEGIN PGP SIG -- END PGP MESSAGE -- with no lines before or after,
+ and rejecting .changes that didn't match that
+
+ o rene should check for source packages not building any binaries
+
+ o heidi should have a diff mode that accepts diff output!
+
+ o halle doesn't deal with melanie'd packages, partial replacements
+ etc. and more.
+
+ o lauren, the tramp, blindly deletes with no check that the delete
+ failed which it might well given we only look for package/version,
+ not package/version _in p-u_. duh.
+
+ o melanie should remove obsolete changes when removing from p-u, or
+ at least warn. or halle should handle it.
+
+ o need a testsuite _badly_
+
+ o lisa should have an Bitch-Then-Accept option
+
+ o jennifer crashes if run as a user in -n mode when orig.tar.gz is
+ in queue/new...
+
+<elmo_home> [<random>maybe I should reject debian packages with a non-Debian origin or bugs field</>]
+<Kamion> [<random>agreed; dunno what origin does but non-Debian bugs fields would be bad]
+
+ o rhona should make use of select..except select, temporary tables
+ etc. rather than looping and calling SQL every time so we can do
+ suite removal sanely (see potato-removal document)
+
+ o melanie will happily include packages in the Cc list that aren't
+ being removed...
+
+ o melanie doesn't remove udebs when removing the source they build from
+
+ o check_dsc_against_db's "delete an entry from files while you're
+ not looking" habit is Evil and Bad.
+
+ o lisa allows you to edit the section and change the component, but
+ really shouldn't.
+
+ o melanie needs to, when not sending bug close mails, promote Cc: to
+ To: and send the mail anyways.
+
+ o the lockfile (Archive_Maintenance_In_Progress) should probably be in a conf file
+
+ o madison should cross-check the b.source field and if it's not null
+ and s.name linked from it != the source given in
+ -S/--source-and-binary ignore.
+
+ o lauren sucks; she should a) only spam d-i for sourceful
+ rejections, b) sort stuff so she rejects sourceful stuff first. the
+ non-sourceful should probably get a form mail, c) automate the
+ non-sourceful stuff (see b).
+
+ o jennifer should do q-d stuff for faster AA [ryan]
+
+ o split the morgue into source and binary so binaries can be purged first!
+
+ o per-architecture priorities for things like different arch'es
+ gcc's, silly BSD libftw, palo, etc.
+
+ o use postgres 7.2's built-in stat features to figure out how indices are used etc.
+
+ o neve shouldn't be using location, she should run down suites instead
+
+ o halle needs to know about udebs
+
+ o by default hamstring katie's mail sending so that she won't send
+ anything until someone edits a script; she's been used far too
+ much to send spam atm :(
+
+ o $ftpdir/indices isn't created by rose because it's not in katie.conf
+
+ o sanity check depends/recommends/suggests too? in fact for any
+ empty field?
- o cross-suite version checking
+[minor] kelly's copychanges, copykatie handling sucks, the per-suite
+ thing is static for all packages, so work out in advance dummy.
- Priority based; stable == 1, proposed-update == 2, etc.
+[madison] # filenames ?
+[madison] # maintainer, component, install date (source only?), fingerprint?
- i.e. A package targeted for a suite, must have a higher (or
- equal) version number than anything in a suite with a lower
- priority and also must have a lower (or equal) version number
- than anything in a suite with a higher priorty. The checks are
- always "or equal to" because two seperate uploads with the same
- version will fail thanks to the file overwrite errors (at least
- ignoring potato) and multi-suite uploads (e.g. "stable
- unstable") need to work.
+ o UrgencyLog stuff should minimize it's bombing out(?)
+ o Log stuff should open the log file
- o jenna sucks
+ o helena should footnote the actual notes, and also * the versions
+ with notes so we can see new versions since being noted...
- o queued and/or perl on sparc stable sucks
+ o helena should have alternative sorting options, including reverse
+ and without or without differentiaion.
+
+ o julia should sync debadmin and ftpmaster (?)
+
+ o <drow> Can't read file.:
+ /org/security.debian.org/queue/accepted/accepted/apache-perl_1.3.9-14.1-1.21.20000309-1_sparc.katie.
+ You assume that the filenames are relative to accepted/, might want
+ to doc or fix that.
+
+ o <neuro> the orig was in NEW, the changes that caused it to be NEW
+ were pulled out in -2, and we end up with no orig in the archive
+ :(
+
+ o SecurityQueueBuild doesn't handle the case of foo_3.3woody1
+ with a new .orig.tar.gz followed by a foo_3.3potato1 with the same
+ .orig.tar.gz; jennifer sees it and copes, but the AA code doesn't
+ and can't really easily know so the potato AA dir is left with no
+ .orig.tar.gz copy. doh.
+
+ o orig.tar.gz in accepted not handled properly (?)
+
+ o amber doesn't include .orig.tar.gz but it should
+
+ o permissions (paranoia, group write, etc.) configurability and overhaul
+
+ o remember duplicate copyrights in lisaand skip them, per package
+
+ o <M>ove option for lisa byhand proecessing
+
+ o rene could do with overrides
+
+ o db_access.get_location_id should handle the lack of archive_id properly
+
+ o the whole versioncmp thing should be documented
+
+ o lisa doesn't do the right thing with -2 and -1 uploads, as you can
+ end up with the .orig.tar.gz not in the pool
+
+ o lisa exits if you check twice (aj)
+
+ o lisa doesn't trap signals from fernanda properly
+
+ o queued and/or perl on sparc stable sucks - reimplement it.
o aj's bin nmu changes
* arch != {any,all}
* build-depends wrong (via andrea)
* suid
- * conficlits
+ * conflicts
* notification/stats to admin daily
o trap fernanda exiting
o distinguish binary only versus others (neuro)
- o change utils.copy to try rename() first
-
o cache changes parsed from ordering (careful tho: would be caching
.changes from world writable incoming, not holding)
who their source is; source-must-exist does, but the info is not
propogated down.
- o DONE
-
- o clean fingerprint
-
o Fix BTS vs. katie sync issues by queueing(via BSMTP) BTS mail so
that it can be released on deman (e.g. ETRN to exim).
- o write our own locking module or in some other way fix things so we
- don't require a patch to python since upstream refuse to fix it.
-
- o maintainers file needs overrides
-
+ o maintainers file needs overrides
+
[ change override.maintainer to override.maintainer-from +
override.maintainer-to and have them reference the maintainers
table. Then fix charisma to use them and write some scripting
to handle the Santiago situation. ]
- o (Depends) <aj> elmo: it should match \(\s*(<<|<|<=|=|>=|>|>>)\s*<VERSIONREGEXP>\) fwiw
+ o Validate Depends (et al.) [it should match \(\s*(<<|<|<=|=|>=|>|>>)\s*<VERSIONREGEXP>\)]
o Clean up DONE; archive to tar file every 2 weeks, update tar tvzf INDEX file.
unstable and source-ver ~= source-ver in testing; then map
unstable -> testing-updates ?
- o [Hard] dependency checking (esp. stable) (partially done)
+ o hooks or configurability for debian specific checks (e.g. check_urgency, auto-building support)
- o hooks for debian specific checks (e.g. check_urgency)
+ o morgue needs auto-cleaning (?)
+
+ o saffron: two modes, all included, seperate
+ o saffron: add non-US
+ o saffron: add ability to control components, architectures, archives, suites
+ o saffron: add key to expand header
- o Logging [mostly done]
+================================================================================
- o Abstraction of suite hardcoding (q.v. progengy)
+queue/approved
+--------------
- o morgue needs auto-cleaning (?)
+ o What to do with multi-suite uploads? Presumably hold in unapproved
+ and warn? Or what? Can't accept just for unstable or reject just
+ from stable.
- o auto-building crap should be optional
+ o Whenever we check for anything in accepted we also need to check in
+ unapproved.
+
+ o non-sourceful uploads should go straight through if they have
+ source in accepted or the archive.
+
+ o security uploads on auric should be pre-approved.
================================================================================
Less Urgent
-----------
- o jennifer is leaking file descriptors..
+ o change utils.copy to try rename() first
- o the tar extractor class doesn't need to be redone for each package
+ o [hard, long term] unchecked -> accepted should go into the db, not
+ a suite, but similar. this would allow katie to get even faster,
+ make madison more useful, decomplexify specialacceptedautobuild
+ and generally be more sane. may even be helpful to have e.g. new
+ in the DB, so that we avoid corner cases like the .orig.tar.gz
+ disappearing 'cos the package has been entirely removed but was
+ still on stayofexecution when it entered new.
- o interrupting of stracing jennifer causes exceptions errors from apt_inst calls
+ o Logging [mostly done] (todo: rhona (hard), .. ?)
+
+ o jennifer: the tar extractor class doesn't need to be redone for each package
o reverse of source-must-exist; i.e. binary-for-source-must-not-exist
o REJECT reminders in shania.
o To incorporate from utils:
@ unreject
- o katie could vrfy email addresses (?) [errge@]
o auto-purge out-of-date stuff from non-free/contrib so that testing and stuff works
o doogie's binary -> source index
- o Add urgency + installed date to the DB for aj/testing (? still needed ?)
o jt's web stuff, matt's changelog stuff (overlap)
o [Hard] Need to merge non-non-US and non-US DBs.
o Improve andrea.
o Need to optimize all the queries by using EXAMINE and building some INDEXs.
+ [postgresql 7.2 will help here]
o Need to enclose all the setting SQL stuff in transactions (mostly done).
o Need to finish alyson (a way to sync katie.conf and the DB)
o Need the ability to rebuild all other tables from dists _or_ pools (in the event of disaster) (?)
@ backup of report (?)
@ backup of changes.tgz (?)
+ o --help doesn't work without /etc/katie/katie.conf (or similar) at
+ least existing.
+
+ o rename andrea (clashes with existing andrea)...
+
* Harder:
+ o interrupting of stracing jennifer causes exceptions errors from apt_inst calls
+ o dependency checking (esp. stable) (partially done)
o override checks sucks; it needs to track changes made by the
maintainer and pass them onto ftpmaster instead of warning the
maintainer.
o need to do proper rfc822 escaping of from lines (as opposed to s/\.//g)
- o when dinstall is run in install mode but not as a cron job, it
- should probably run jenna to avoid madison originated user confusion
o Revisit linking of binary->source in install() in katie.
o Fix component handling in overrides (aj)
o Fix lack of entires in source overrides (aj)
- o direport misreports things as section 'devel'
+ o direport misreports things as section 'devel' (? we don't use direport)
o vrfy check of every Maintainer+Changed-By address; valid for 3 months.
o binary-all should be done on a per-source, per-architecture package
basis to avoid, e.g. the perl-modules problem.
o katie/jenna/rhona/whatever needs to clear out .changes
files from p-u when removing stuff superseded by newer versions.
[but for now we have halle]
- o test sig checking stuff in test/ (stupid thing is not modularized)
+ o test sig checking stuff in test/ (stupid thing is not modularized due to global abuse)
o when encountering suspicous things (e.g. file tainting) do something more drastic
* Easy:
+ o suite mapping and component mapping are parsed per changes file,
+ they should probably be stored in a dictionary created at startup.
+ o don't stat/md5sum files you have entries for in the DB, moron
+ boy (Katie.check_source_blah_blah)
o promote changes["changes"] to mandatory in katie.py(dump_vars)
after a month or so (or all .katie files contain in the queue
contain it).
- o Rationalize config stuff.. Dir::Foo not Dir::FooDir [breaks other people??]
- Also Dir::Queue::Foo, not Dir::QueueFooDir
o melanie should behave better with -a and without -b; see
gcc-defaults removal for an example.
o Reject on misconfigured kernel-package uploads
- o denise abuses sys.stdout == badly
o utils.extract_component_from_section: main/utils -> main/utils, main rather than utils, main
- o Fix katie to warn if run when not in incoming or p-u
- o check for empty debs; check for empty (or small (<1k ?)) .tar.gz's.
+ o Fix jennier to warn if run when not in incoming or p-u
o katie should validate multi-suite uploads; only possible valid one
is "stable unstable"
o cron.daily* should change umask (aj sucks)
o Rene doesn't look at debian-installer but should.
o Rene needs to check for binary-less source packages.
o Rene could accept a suite argument (?)
- o run shania every day (?) [ryan]
- o we don't check for .orig.tar.gz's, so it's possible to upload -2 with no .orig.tar.gz.
o byhand stuff should send notification
o catherine should udpate db; move files, not the other way around [neuro]
o melanie should update the stable changelog [joey]
o update tagdb.dia
-
+
* Bizzare/uncertain:
-
- o validate source consistency, i.e. .dsc and .tar.gz as minimum (??)
+
+ o drop rather dubious currval stuff (?)
+ o rationalize os.path.join() usage
o Rene also doesn't seem to warn about missing binary packages (??)
- o jenna should not delete things because they don't exist (?)
o logging: hostname + pid ?
o ANAIS should be done in katie (?)
o Add an 'add' ability to melanie (? separate prog maybe)
o Handle the case of 1:1.1 which would overwrite 1.1 (?)
o maybe drop -r/--regex in madison, make it the default and
implement -e/--exact (a la joey's "elmo")
-
- * Cleanups & misc:
-
- o db_access' get_files needs to use exceptions not this None, > 0, < 0 return val BS (?)
- o The untouchable flag doesn't stop new packages being added to ``untouchable'' suites
- o jenna is too slow
- o jenna doesn't handle arch: any -> arch: all transitions
- [aj worked around; need to revisit]
+ o dsc files are not checked for existence/perms (only an issue if
+ they're in the .dsc, but not the .changes.. possible?)
- * Stable:
+ * Cleanups & misc:
-<neuro> well, there's one other thing that doesn't work so well
-<neuro> you probably should have stable_install ignore Closes:
-<neuro> or it might be unique to the case of stable unstable uploads
+ o db_access' get_files needs to use exceptions not this None, > 0, < 0 return val BS (?)
+ o The untouchable flag doesn't stop new packages being added to ``untouchable'' suites
================================================================================
o Fix stuff to look in sensible places for libs and config file in debian package (?)
+================================================================================
+
+ --help manpage
+-----------------------------------------------------------------------------
+alyson X
+amber X
+andrea X
+ashley X
+catherine X X
+charisma X X
+cindy X X
+claire X
+denise X
+fernanda X
+halle X
+heidi X X
+helena X
+jenna X
+jennifer X
+jeri X
+julia X X
+kelly X X
+lisa X X
+madison X X
+melanie X X
+natalie X X
+neve X
+rene X
+rose X
+rhona X X
+saffron X
+shania X
+tea X
+ziyi X
+
+
+================================================================================
+
+Random useful-at-some-point SQL
+-------------------------------
+
+UPDATE files SET last_used = '1980-01-01'
+ FROM binaries WHERE binaries.architecture = <x>
+ AND binaries.file = files.id;
+
+DELETE FROM bin_associations
+ WHERE EXISTS (SELECT id FROM binaries
+ WHERE architecture = <x>
+ AND id = bin_associations.bin);
+
================================================================================
projects / dak.git / blobdiff