billie: Cleaned up a load of comments, added /README.non-US tothe verbatim matches...

[dak.git] / TODO

                                 TODO
                                 ====

[NB: I use this as a thought record/scribble, not everything on here
     makes sense and/or is actually ever going to get done, so IIWY I
     wouldn't use it as gospel for the future of katie or as a TODO
     list for random hacking.]

================================================================================

queue/approved
--------------

 o What to do with multi-suite uploads?  Presumably hold in unapproved
   and warn?  Or what?  Can't accept just for unstable or reject just
   from stable.

 o Whenever we check for anything in accepted we also need to check in
   unapproved.

 o non-sourceful uploads should go straight through if they have
   source in accepted or the archive.

 o security uploads on auric should be pre-approved.


Others
------

  o need to decide on whether we're tying for most errors at once.. if
    so (probably) then make sure code doesn't assume variables exist and
    either way do something about checking error code of check_dsc and
    later functions so we skip later checks if they're bailing.

  o the .katie stuff is fundamentally braindamaged, it's not versioned
    so there's no way to change the format, yay me.  need to fix.
    probably by putting a version var as the first thing and checking
    that.. auto-upgrade at least from original format would be good.
    might also be a good idea to put everything in one big dict after
    that?

  o [?, wishlist, distant future] RFC2047-ing should be extended to
    all headers of mails sent out.

  o reject sparc64 binaries in a non '*64*' package.

  o katie.py(source_exists): a) we take arguments as parameters that
    we could figure out for ourselves (we're part of the Katie class
    after all), b) we have this 3rd argument which defaults to "any"
    but could in fact be dropped since no one uses it like that.

  o jennifer: doesn't handle bin-only NMUs of stuff still in NEW,
    BYHAND or ACCEPTED (but not the pool) - not a big deal, upload can
    be retried once the source is in the archive, but still.

  o security global mail overrides should special case buildd stuff so
    that buildds get ACCEPTED mails (or maybe amber (?)), that way
    upload-security doesn't grow boundlessly.

  o amber should upload sourceful packages first, otherwise with big
    packages (e.g. X) and esp. when source is !i386, half the arches
    can be uploaded without source, get copied into queue/unaccepted
    and promptly rejected.

  o rene's NVIU check doesn't catch cases where source package changed
    name, should check binaries too. [debian-devel@l.d.o, 2004-02-03]

  o cnf[melanie::logfile] is misnamed...

<aj> i'd be kinda inclined to go with insisting the .changes file take
   the form ---- BEGIN PGP MESSAGE --- <non -- BEGIN/END lines> --
   BEGIN PGP SIG -- END PGP MESSAGE -- with no lines before or after,
   and rejecting .changes that didn't match that

  o rene should check for source packages not building any binaries

  o heidi should have a diff mode that accepts diff output!

  o halle doesn't deal with melanie'd packages, partial replacements
   etc. and more.

  o lauren, the tramp, blindly deletes with no check that the delete
   failed which it might well given we only look for package/version,
   not package/version _in p-u_.  duh.

  o melanie should remove obsolete changes when removing from p-u, or
   at least warn.  or halle should handle it.

  o need a testsuite _badly_

  o lisa should have an Bitch-Then-Accept option

  o jennifer crashes if run as a user in -n mode when orig.tar.gz is
   in queue/new...

<elmo_home> [<random>maybe I should reject debian packages with a non-Debian origin or bugs field</>]
<Kamion> [<random>agreed; dunno what origin does but non-Debian bugs fields would be bad]

  o rhona should make use of select..except select, temporary tables
  etc. rather than looping and calling SQL every time so we can do
  suite removal sanely (see potato-removal document)

  o melanie will happily include packages in the Cc list that aren't
    being removed...

  o melanie doesn't remove udebs when removing the source they build from

  o check_dsc_against_db's "delete an entry from files while you're
    not looking" habit is Evil and Bad.

  o lisa allows you to edit the section and change the component, but
    really shouldn't.

  o melanie needs to, when not sending bug close mails, promote Cc: to
    To: and send the mail anyways.

  o the lockfile (Archive_Maintenance_In_Progress) should probably be in a conf file

  o madison should cross-check the b.source field and if it's not null
    and s.name linked from it != the source given in
    -S/--source-and-binary ignore.

  o lauren sucks; she should a) only spam d-i for sourceful
  rejections, b) sort stuff so she rejects sourceful stuff first.  the
  non-sourceful should probably get a form mail, c) automate the
  non-sourceful stuff (see b).

  o jennifer should do q-d stuff for faster AA [ryan]

  o split the morgue into source and binary so binaries can be purged first!

  o per-architecture priorities for things like different arch'es
    gcc's, silly BSD libftw, palo, etc.

  o use postgres 7.2's built-in stat features to figure out how indices are used etc.

  o neve shouldn't be using location, she should run down suites instead

  o halle needs to know about udebs

  o by default hamstring katie's mail sending so that she won't send
    anything until someone edits a script; she's been used far too
    much to send spam atm :(

  o $ftpdir/indices isn't created by rose because it's not in katie.conf

  o sanity check depends/recommends/suggests too?  in fact for any
    empty field?

[minor] kelly's copychanges, copykatie handling sucks, the per-suite
        thing is static for all packages, so work out in advance dummy.

[madison] # filenames ?
[madison] # maintainer, component, install date (source only?), fingerprint?

  o UrgencyLog stuff should minimize it's bombing out(?)
  o Log stuff should open the log file

  o helena should footnote the actual notes, and also * the versions
    with notes so we can see new versions since being noted...

  o helena should have alternative sorting options, including reverse
    and without or without differentiaion.

  o julia should sync debadmin and ftpmaster (?)

  o <drow> Can't read file.:
  /org/security.debian.org/queue/accepted/accepted/apache-perl_1.3.9-14.1-1.21.20000309-1_sparc.katie.
  You assume that the filenames are relative to accepted/, might want
  to doc or fix that.

  o <neuro> the orig was in NEW, the changes that caused it to be NEW
    were pulled out in -2, and we end up with no orig in the archive
    :(

  o SecurityAcceptedAutoBuild doesn't handle the case of foo_3.3woody1
    with a new .orig.tar.gz followed by a foo_3.3potato1 with the same
    .orig.tar.gz; jennifer sees it and copes, but the AA code doesn't
    and can't really easily know so the potato AA dir is left with no
    .orig.tar.gz copy.  doh.

  o orig.tar.gz in accepted not handled properly (?)

  o amber doesn't include .orig.tar.gz but it should

  o permissions (paranoia, group write, etc.) configurability and overhaul

  o remember duplicate copyrights in lisaand skip them, per package

  o <M>ove option for lisa byhand proecessing

  o rene could do with overrides

  o db_access.get_location_id should handle the lack of archive_id properly

  o the whole versioncmp thing should be documented

  o lisa doesn't do the right thing with -2 and -1 uploads, as you can
    end up with the .orig.tar.gz not in the pool

  o lisa exits if you check twice (aj)

  o lisa doesn't trap signals from fernanda properly

  o queued and/or perl on sparc stable sucks - reimplement it.

  o aj's bin nmu changes

  o Lisa:
    * priority >> optional
    * arch != {any,all}
    * build-depends wrong (via andrea)
    * suid
    * conficlits
    * notification/stats to admin daily
    o trap fernanda exiting
    o distinguish binary only versus others (neuro)

  o cache changes parsed from ordering (careful tho: would be caching
    .changes from world writable incoming, not holding)

  o katie doesn't recognise binonlyNMUs correctly in terms of telling
    who their source is; source-must-exist does, but the info is not
    propogated down.

  o Fix BTS vs. katie sync issues by queueing(via BSMTP) BTS mail so
    that it can be released on deman (e.g. ETRN to exim).

  o maintainers file needs overrides

    [ change override.maintainer to override.maintainer-from +
      override.maintainer-to and have them reference the maintainers
      table.  Then fix charisma to use them and write some scripting
      to handle the Santiago situation. ]

  o Validate Depends (et al.) [it should match  \(\s*(<<|<|<=|=|>=|>|>>)\s*<VERSIONREGEXP>\)]

  o Clean up DONE; archive to tar file every 2 weeks, update tar tvzf INDEX file.

  o testing-updates suite: if binary-only and version << version in
    unstable and source-ver ~= source-ver in testing; then map
    unstable -> testing-updates ?

  o hooks or configurability for debian specific checks (e.g. check_urgency, auto-building support)

  o morgue needs auto-cleaning (?)

  o saffron: two modes, all included, seperate
  o saffron: add non-US
  o saffron: add ability to control components, architectures, archives, suites
  o saffron: add key to expand header

================================================================================

Less Urgent
-----------

  o change utils.copy to try rename() first

  o [hard, long term] unchecked -> accepted should go into the db, not
    a suite, but similar.  this would allow katie to get even faster,
    make madison more useful, decomplexify specialacceptedautobuild
    and generally be more sane.  may even be helpful to have e.g. new
    in the DB, so that we avoid corner cases like the .orig.tar.gz
    disappearing 'cos the package has been entirely removed but was
    still on stayofexecution when it entered new.

  o Logging [mostly done] (todo: rhona (hard), .. ?)

  o jennifer: the tar extractor class doesn't need to be redone for each package

  o reverse of source-must-exist; i.e. binary-for-source-must-not-exist
  o REJECT reminders in shania.
  o fernanda should check for conflicts and warn about them visavis priority [rmurray]
  o store a list of removed/files versions; also compare against them.
    [but be careful about scalability]

  o fernanda: print_copyright should be a lot more intelligent
     @ handle copyright.gz
     @ handle copyright.ja and copyright
     @ handle (detect at least) symlinks to another package's doc directory
     @ handle and/or fall back on source files (?)

  o To incorporate from utils:
     @ unreject

  o auto-purge out-of-date stuff from non-free/contrib so that testing and stuff works
  o doogie's binary -> source index
  o jt's web stuff, matt's changelog stuff (overlap)

  o [Hard] Need to merge non-non-US and non-US DBs.

  o experimental needs to auto clean (relative to unstable) [partial: rene warns about this]

  o Do a checkpc(1)-a-like which sanitizes a config files.
  o fix parse_changes()/build_file_list() to sanity check filenames
  o saftey check and/or rename debs so they match what they should be

  o Improve andrea.
  o Need to optimize all the queries by using EXAMINE and building some INDEXs.
    [postgresql 7.2 will help here]
  o Need to enclose all the setting SQL stuff in transactions (mostly done).
  o Need to finish alyson (a way to sync katie.conf and the DB)
  o Need the ability to rebuild all other tables from dists _or_ pools (in the event of disaster) (?)
  o Make the --help and --version options do stuff for all scripts

  o charisma can't handle whitespace-only lines (for the moment, this is feature)

  o generic way of saying isabinary and isadsc. (?)

  o s/distribution/suite/g

  o cron.weekly:
     @ weekly postins to d-c (?)
     @ backup of report (?)
     @ backup of changes.tgz (?)

  o --help doesn't work without /etc/katie/katie.conf (or similar) at
    least existing.

  o rename andrea (clashes with existing andrea)...

 * Harder:

    o interrupting of stracing jennifer causes exceptions errors from apt_inst calls
    o dependency checking (esp. stable) (partially done)
    o override checks sucks; it needs to track changes made by the
      maintainer and pass them onto ftpmaster instead of warning the
      maintainer.
    o need to do proper rfc822 escaping of from lines (as opposed to s/\.//g)
    o Revisit linking of binary->source in install() in katie.
    o Fix component handling in overrides (aj)
    o Fix lack of entires in source overrides (aj)
    o direport misreports things as section 'devel' (? we don't use direport)
    o vrfy check of every Maintainer+Changed-By address; valid for 3 months.
    o binary-all should be done on a per-source, per-architecture package
      basis to avoid, e.g. the perl-modules problem.
    o a source-missing-diff check: if the version has a - in it, and it
      is sourceful, it needs orig and diff, e.g. if someone uploads
      esound_0.2.22-6, and it is sourceful, and there is no diff ->
      REJECT (version has a dash, therefore not debian native.)
    o check linking of .tar.gz's to .dsc's.. see proftpd 1.2.1 as an example
    o archive needs md5sum'ed regularly, but takes too long to do all
      in one go; make progressive or weekly.
    o katie/jenna/rhona/whatever needs to clear out .changes
      files from p-u when removing stuff superseded by newer versions.
      [but for now we have halle]
    o test sig checking stuff in test/ (stupid thing is not modularized due to global abuse)
    o when encountering suspicous things (e.g. file tainting) do something more drastic

 * Easy:

    o suite mapping and component mapping are parsed per changes file,
      they should probably be stored in a dictionary created at startup.
    o don't stat/md5sum files you have entries for in the DB, moron
      boy (Katie.check_source_blah_blah)
    o promote changes["changes"] to mandatory in katie.py(dump_vars)
      after a month or so (or all .katie files contain in the queue
      contain it).
    o melanie should behave better with -a and without -b; see
      gcc-defaults removal for an example.
    o Reject on misconfigured kernel-package uploads
    o utils.extract_component_from_section: main/utils -> main/utils, main rather than utils, main
    o Fix jennier to warn if run when not in incoming or p-u
    o katie should validate multi-suite uploads; only possible valid one
      is "stable unstable"
    o cron.daily* should change umask (aj sucks)
    o Rene doesn't look at debian-installer but should.
    o Rene needs to check for binary-less source packages.
    o Rene could accept a suite argument (?)
    o byhand stuff should send notification
    o catherine should udpate db; move files, not the other way around [neuro]
    o melanie should update the stable changelog [joey]
    o update tagdb.dia

 * Bizzare/uncertain:

    o drop rather dubious currval stuff (?)
    o rationalize os.path.join() usage
    o Rene also doesn't seem to warn about missing binary packages (??)
    o logging: hostname + pid ?
    o ANAIS should be done in katie (?)
    o Add an 'add' ability to melanie (? separate prog maybe)
    o Replicate old dinstall report stuff (? needed ?)
    o Handle the case of 1:1.1 which would overwrite 1.1 (?)
    o maybe drop -r/--regex in madison, make it the default and
      implement -e/--exact (a la joey's "elmo")
    o dsc files are not checked for existence/perms (only an issue if
      they're in the .dsc, but not the .changes.. possible?)

 * Cleanups & misc:

    o db_access' get_files needs to use exceptions not this None, > 0, < 0 return val BS (?)
    o The untouchable flag doesn't stop new packages being added to ``untouchable'' suites

================================================================================

Packaging
---------

  o Fix stuff to look in sensible places for libs and config file in debian package (?)

================================================================================

           --help      manpage
-----------------------------------------------------------------------------
alyson        X
amber         X
andrea        X
ashley        X
catherine     X          X
charisma      X          X
cindy         X          X
claire        X
denise        X
fernanda      X
halle         X
heidi         X          X
helena        X
jenna         X
jennifer      X
jeri          X
julia         X          X
kelly         X          X
lisa          X          X
madison       X          X
melanie       X          X
natalie       X          X
neve          X
rene          X
rose          X
rhona         X          X
saffron       X
shania        X
tea           X
ziyi          X

================================================================================

Possible Quotes:
----------------
 
| <mdz_> SirDibos: that sentence sounds like it wants to be a bug report when it grows up
 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

| From: Andrew Morton <akpm@osdl.org>
| Subject: Re: Linux 2.6.0-test1 Ext3 Ooops. Reboot needed.
| To: Ricardo Galli <gallir@uib.es>
| Cc: linux-kernel@vger.kernel.org
| Date: Fri, 18 Jul 2003 14:27:20 -0700
| 
| Ricardo Galli <gallir@uib.es> wrote:
| >
| > "File alteration monitor", from Debian.
| 
| OK.
| 
| > $ apt-cache show fam
| 
| I was attacked by dselect as a small child and have since avoided debian. 
| Is there a tarball anywhere?

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

| From: Bob Hilliard <hilliard@debian.org>
| 
|      In my experience, James has been very responsive, albeit not
| verbose, to reasonable questions/requests that don't start out saying
| "James is a bum - throw him out".

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

| <eigood> Kamion: are you too busy to look at my generic
|          include/exclude stuff for the bts yet?
| <Kamion> eigood: expect me to be busy for about the next week at this
|          rate
| <eigood> my %field_match = (
| <eigood>     'subject' => \&contains_field_match,
| <eigood>     'severity' => \&exact_field_match,
| <eigood> that's how it works, basically
| <eigood> I'm a big fan of callbacks
| [...]
| <eigood> Kamion: how do you feel about having
|          per-bug/per-package/per-source notes support in the bts?
| <Kamion> eigood: as I said five minutes ago, I really don't have time
|          to think about it right now, sorry
| <Kamion> here, maybe it would be clearer if I /part
| <-- Kamion (~cjwatson@host81-129-36-235.in-addr.btopenworld.com) has left #debian-devel (too busy. no, really.)
| <eigood> no need to be hostile
| <Joy> eigood: he told you he's too busy and you kept bugging him. take
|       a hint :)

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

<mstone> bwahahaha. Dear "security@debian.org" Thank you for your
         email about "[SECURITY] [DSA-403-1] userland can access Linux
         kernel memory" ...I need to filter out spam... To send email to
         vhs@flexdesign.com please put "ducks" anywhere on your subject
         line. ...Thanks, Bob...
<mstone> I'll be sure to do that...

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

<drow> Hmm, that was a nice short bug report.
<drow> to submit@: "strdup(NULL) segfaults" to -done@: "Yes, go away"
[...]
<Kamion> how did he pass T&S? sheer bloody-mindedness?
[...]
<drow> Good attention to detail?
<drow> Masking of psychopathic tendencies?
* drow shrugs

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

<DanielS> the people love me
<Joy> like pneumonia

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

test.c:5: `long long long' is too long for GCC

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

http://yro.slashdot.org/comments.pl?sid=91696&cid=7890274

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

<Joy> argh.
<Joy> i accidentally banned all mails to the bts that had 'ossi' in them
<Joy> "possible" etc

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

<http://www.livejournal.com/users/mjg59/2003/12/24/>

Wednesday, December 24th, 2003
3:34 pm         
Dear PC World,

1) The most common chipset used in Pentium-II machines is the Intel 440BX. It is also relatively common in slower P-IIIs, and is approximately identical to the 440MX (a one-chip version aimed at laptops).

2) The 440BX has the interesting feature of only being able to address up to 128MBit density RAM. This is a relatively widely known issue.

3) Simple maths suggests that if you have a 128MB DIMM with 4 chips on it, they are likely to be 256MBit parts.

4) Marking said DIMMs as being suitable for Pentium-IIs is therefore really indescribably stupid, you wankwits. Please fuck off and die in a great big chemical fire before I get back there to beat you.

Love,

Matthew.

PS,

Die. No, really. 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

<http://www.livejournal.com/users/mjg59/2003/11/12/>

Wednesday, November 12th, 2003
2:43 am         
It's true that you learn something new every day. Yesterday I discovered that playdough is electrically conductive. I also discovered that RAM becomes unhappy if all of its pins are joined together with electrically conductive material.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

<http://www.livejournal.com/users/mjg59/2003/11/03/>

Monday, November 3rd, 2003
3:13 pm         
Hint to people attempting to sell things online:

DON'T PUT http://172.16.100.107/ IN YOUR URLS, YOU INCOMPETENT FUCKMONKEYS 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

| priviledged positions? What privilege? The honour of working harder
| than most people for absolutely no recognition?

Manoj Srivastava <srivasta@debian.org> in <87lln8aqfm.fsf@glaurung.internal.golden-gryphon.com>

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

<elmo_h> you could just r00t klecker through [...] and do it yourself
<mdz> heh
<mdz> I think there's a bit in the DMUP about that

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

<Yoe> well, thing is, he doesn't seem to understand you usually don't
      have the time to give everyone status updates when a fly moves a
      leg

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

In Soviet Russia...

The cops arrest YOU for not showing papers. Wait, I didn't have to
reverse it this time, what's going on?

http://slashdot.org/comments.pl?sid=97501&cid=8334726

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

<infinity> <shrug>... Messaging IRCops isn't the end of the world,
           unless its "/msg ircop I fucked your wife."

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

<aj> ooo, elmo has "special powers"
<neuro> ooo, does he have lasers that shoot out of his eyes?
<aj> dunno
<aj> maybe he can turn invisible? that'd sure help with improved transparency!

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

| From: Andrew Morton <akpm@osdl.org>
| Subject: Re: [PATCH] Compile kernel with GCC-3.5 and without regparm
| To: "Art Haas" <ahaas@airmail.net>
| Cc: linux-kernel@vger.kernel.org
| Date: Tue, 2 Mar 2004 16:59:28 -0800
| X-Mailer: Sylpheed version 0.9.7 (GTK+ 1.2.10; i586-pc-linux-gnu)
| 
| "Art Haas" <ahaas@airmail.net> wrote:
| >
| > I tried to build the kernel with my CVS GCC-3.5 compiler today, and had
| > all sorts of failures about prototypes not matching.
| 
| -mm is where the gcc-3.5 action is.  There seems to be a bit of an arms
| race going on wherein the gcc developers are trying to break the kernel
| build faster than I and others can fix it.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

(Note that the above is a gross oversimplification, and ignores issues
including but not necessarily limited to subarchitectures, and quality
of hardware coverage within certian architectures. It contains forward
looking statements, and may cause cancer in lab animals.)

Joey Hess in <20040317065216.GA29816@kitenet.net>

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

<jdub> now there's a thought
<jdub> DD trading cards
<mdz> official joeyh action figure, with rapid-fire upload action
<jdub> lamont with pump-action NMU flame-thrower!

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%