X-Git-Url: https://git.decadent.org.uk/gitweb/?a=blobdiff_plain;f=TODO;h=a7213f451dc3afe00858485b46618c2016737443;hb=f02f2fe4b139d08867a090e962b8431ec986ebdf;hp=4c35c4055331cf1620b64773c6c9c2eb9b6af9ce;hpb=15046787dd73ccc528fbae8a89bdec63507f2bb9;p=dak.git

diff --git a/TODO b/TODO
index 4c35c405..a7213f45 100644
--- a/TODO
+++ b/TODO
@@ -1,18 +1,132 @@
-TOFIX
-=====
+				 TODO
+				 ====
 
-Urgent
+[NB: I use this as a thought record/scribble, not everything on here
+     makes sense and/or is actually ever going to get done, so IIWY I
+     wouldn't use it as gospel for the future of katie or as a TODO
+     list for random hacking.]
+
+================================================================================
+
+queue/approved
+--------------
+
+ o What to do with multi-suite uploads?  Presumably hold in unapproved
+   and warn?  Or what?  Can't accept just for unstable or reject just
+   from stable.
+
+ o Whenever we check for anything in accepted we also need to check in
+   unapproved.
+
+ o non-sourceful uploads should go straight through if they have
+   source in accepted or the archive.
+
+ o security uploads on auric should be pre-approved.
+
+
+Others
 ------
 
-  o rewrite shania to a) be sane, b) clean REJECT properly
-    (i.e. ignore the fact that stuff is in a valid .changes file and
-    just clean the mofo)
+  o use postgres 7.2's built-in stat features to figure out how indices are used etc.
+
+  o neve shouldn't be using location, she should run down suites instead
+
+  o halle needs to know about udebs
+
+  o by default hamstring katie's mail sending so that she won't send
+    anything until someone edits a script; she's been used far too
+    much to send spam atm :(
+
+  o $ftpdir/indices isn't created by rose because it's not in katie.conf
+
+  o sanity check depends/recommends/suggests too?  in fact for any
+    empty field?
+
+[minor] kelly's copychanges, copykatie handling sucks, the per-suite
+        thing is static for all packages, so work out in advance dummy.
+
+[madison] # filenames ?
+[madison] # maintainer, component, install date (source only?), fingerprint?
+
+  o UrgencyLog stuff should minimize it's bombing out(?)
+  o Log stuff should open the log file
+
+  o [melanie] bomb out if a) no '[rene]' in the -m/--message, and b)
+    no other -d/--done or -C/--carbon-copy
+
+  o helena should footnote the actual notes, and also * the versions
+    with notes so we can see new versions since being noted...
+
+  o helena should have alternative sorting options, including reverse
+    and without or without differentiaion.
+
+  o julia should sync debadmin and ftpmaster (?)
+
+  o <drow> Can't read file.:
+  /org/security.debian.org/queue/accepted/accepted/apache-perl_1.3.9-14.1-1.21.20000309-1_sparc.katie.
+  You assume that the filenames are relative to accepted/, might want
+  to doc or fix that.
+
+  o <neuro> the orig was in NEW, the changes that caused it to be NEW
+    were pulled out in -2, and we end up with no orig in the archive
+    :(
+  
+  o SecurityAcceptedAutoBuild doesn't handle the case of foo_3.3woody1
+    with a new .orig.tar.gz followed by a foo_3.3potato1 with the same
+    .orig.tar.gz; jennifer sees it and copes, but the AA code doesn't
+    and can't really easily know so the potato AA dir is left with no
+    .orig.tar.gz copy.  doh.
+
+  o orig.tar.gz in accepted not handled properly (?)
+
+  o amber doesn't include .orig.tar.gz but it should
+
+  o permissions (paranoia, group write, etc.) configurability and overhaul
+
+  o remember duplicate copyrights in lisaand skip them, per package
+
+  o <M>ove option for lisa byhand proecessing
+
+  o rene could do with overrides
+ 
+  o db_access.get_location_id should handle the lack of archive_id properly
+
+  o the whole versioncmp thing should be documented
+
+  o lisa doesn't do the right thing with -2 and -1 uploads, as you can
+    end up with the .orig.tar.gz not in the pool
+
+  o lisa exits if you check twice (aj)
+
+  o lisa doesn't trap signals from fernanda properly
+
+  o queued and/or perl on sparc stable sucks - reimplement it.
+
+  o aj's bin nmu changes
+
+  o Lisa:
+    * priority >> optional
+    * arch != {any,all}
+    * build-depends wrong (via andrea)
+    * suid
+    * conficlits
+    * notification/stats to admin daily
+    o trap fernanda exiting
+    o distinguish binary only versus others (neuro)
 
-  o fix katie's handling of source packages in different directories;
-    e.g. if a package moves from main to contrib, the .orig.tar.gz
-    needs to what? copy or be moved and old source removed; one of the
-    two.
+  o cache changes parsed from ordering (careful tho: would be caching
+    .changes from world writable incoming, not holding)
 
+  o katie doesn't recognise binonlyNMUs correctly in terms of telling
+    who their source is; source-must-exist does, but the info is not
+    propogated down.
+
+  o Fix BTS vs. katie sync issues by queueing(via BSMTP) BTS mail so
+    that it can be released on deman (e.g. ETRN to exim).
+
+  o write our own locking module or in some other way fix things so we
+    don't require a patch to python since upstream refuse to fix it.
+ 
   o maintainers file needs overrides  
  
     [ change override.maintainer to override.maintainer-from +
@@ -20,91 +134,206 @@ Urgent
       table.  Then fix charisma to use them and write some scripting
       to handle the Santiago situation. ]
 
-  o Add urgency + installed date to the DB for aj/testing (? still needed ?)
+  o Validate Depends (et al.) [it should match  \(\s*(<<|<|<=|=|>=|>|>>)\s*<VERSIONREGEXP>\)]
+
+  o Clean up DONE; archive to tar file every 2 weeks, update tar tvzf INDEX file.
+
+  o testing-updates suite: if binary-only and version << version in
+    unstable and source-ver ~= source-ver in testing; then map
+    unstable -> testing-updates ?
 
-  o Disallow binary versions with no source (both newer and older)
-    except for bin-only NMUs (but: have to change sorting so that
-    sourceful uploads are processed first).
+  o hooks or configurability for debian specific checks (e.g. check_urgency, auto-building support)
 
-  o Revisit linking of binary->source in install() in katie.
+  o morgue needs auto-cleaning (?)
+
+  o saffron: two modes, all included, seperate
+  o saffron: add non-US
+  o saffron: add ability to control components, architectures, archives, suites
+  o saffron: add key to expand header
+
+================================================================================
 
 Less Urgent
 -----------
 
-  o configurize all the hardcoded Bcc's in mail templates
-  o melanie (and possibly others) hardcode @*debian.org all over the shop
-  o melanie should warn/error on -C without -d
+  o Accept "," as a seperator in -a, -s etc. args
 
-  o ziyi shouldn't really do stable
+  o change utils.copy to try rename() first
 
-  o need to have process_it run a big try: loop so that individual
-    package screws up don't kill the daily cron run.
+  o [hard, long term] unchecked -> accepted should go into the db, not
+    a suite, but similar.  this would allow katie to get even faster,
+    make madison more useful, decomplexify specialacceptedautobuild
+    and generally be more sane.  may even be helpful to have e.g. new
+    in the DB, so that we avoid corner cases like the .orig.tar.gz
+    disappearing 'cos the package has been entirely removed but was
+    still on stayofexecution when it entered new.
 
-  o Add an 'add' ability to melanie (? separate prog maybe)
-  o [Hard] Need to merge non-non-US and non-US DBs.
-  o [Easy] Need to vacuum daily
-  o [Easy] remove empty directories in the pool in cron.weekly or so
+  o Logging [mostly done] (todo: rhona (hard), .. ?)
 
-  o Fix dinstall to remove != stable on stable install, not just f + u
-  o Fix 'mixed' component handling
-  o Fix component handling in overrides (aj)
-  o Fix lack of entires in source overrides (aj)
-  o Fix rhona to unconditionally read override file on startup
-  o Fix dinstall to warn if not in incoming or p-u
-  o db_access' get_files needs to use exceptions not this None, > 0, < 0 return val BS (?)
-  o check errors on apt_pkg calls (?)
-  o The untouchable flag doesn't stop adding new packages
-  o jenna needs to validate arguments (?)
+  o jennifer: the tar extractor class doesn't need to be redone for each package
 
-===================================================================================================
+  o reverse of source-must-exist; i.e. binary-for-source-must-not-exist
+  o REJECT reminders in shania.
+  o fernanda should check for conflicts and warn about them visavis priority [rmurray]
+  o store a list of removed/files versions; also compare against them.
+    [but be careful about scalability]
 
-TODO
-====
+  o fernanda: print_copyright should be a lot more intelligent
+     @ handle copyright.gz
+     @ handle copyright.ja and copyright
+     @ handle (detect at least) symlinks to another package's doc directory
+     @ handle and/or fall back on source files (?)
 
-Packaging
----------
+  o To incorporate from utils:
+     @ unreject
 
-  o Install python libraries (db_access and utils)
-  o Install config file
-  o Fix stuff to look in sensible places for libs and config file in debian package (?)
-  o man pages and/or documentation
+  o auto-purge out-of-date stuff from non-free/contrib so that testing and stuff works
+  o doogie's binary -> source index
+  o jt's web stuff, matt's changelog stuff (overlap)
 
-Urgent
-------
+  o [Hard] Need to merge non-non-US and non-US DBs.
 
-  o Secure incoming handling
-  o [Hard] dependency checking
-  o dpkg 1.7 Changed-By field support
-  o Overrides for Maintainers
-  o Kill orphaned
+  o experimental needs to auto clean (relative to unstable) [partial: rene warns about this]
 
-Less Urgent
------------
+  o Do a checkpc(1)-a-like which sanitizes a config files.
+  o fix parse_changes()/build_file_list() to sanity check filenames
+  o saftey check and/or rename debs so they match what they should be
 
-  o Enable catherine (when daily mirror hit drops down below 300Mb maybe..)
   o Improve andrea.
-  o Need a poolifier that will poolify X mb a day.. (catherine)
   o Need to optimize all the queries by using EXAMINE and building some INDEXs.
+    [postgresql 7.2 will help here]
   o Need to enclose all the setting SQL stuff in transactions (mostly done).
   o Need to finish alyson (a way to sync katie.conf and the DB)
-  o Need log files for jenna, rhona and katie.
   o Need the ability to rebuild all other tables from dists _or_ pools (in the event of disaster) (?)
   o Make the --help and --version options do stuff for all scripts
-  o Need to check for .dsc when source is mentioned
-
-  o fix parse_changes()/build_file_list() to sanity check filenames
-  o saftey check and/or rename debs so they match what they should be
 
   o charisma can't handle whitespace-only lines (for the moment, this is feature)
-  o Should use $EDITOR, not hardcode vi
-  o Should reject packages with bad timestamps
-  o Replicate old dinstall report stuff (? needed ?)
-  o Handle the case of 1:1.1 which would overwrite 1.1 (?)
-  o aj's binary-all stuff (foo-doc depending on foo) (?)
-  o heidi should report suite name not ID [aj]
-
-  o fubar and warn/error wrappers like in C (?)
+
   o generic way of saying isabinary and isadsc. (?)
-  o substitution stuff (cf. userdir-ldap) for announce, reject etc.
 
   o s/distribution/suite/g
+
+  o cron.weekly:
+     @ weekly postins to d-c (?)
+     @ backup of report (?)
+     @ backup of changes.tgz (?)
+
+  o --help doesn't work without /etc/katie/katie.conf (or similar) at
+    least existing.
+
+  o rename andrea (clashes with existing andrea)...
+
+ * Harder:
+
+    o interrupting of stracing jennifer causes exceptions errors from apt_inst calls 
+    o dependency checking (esp. stable) (partially done)
+    o override checks sucks; it needs to track changes made by the
+      maintainer and pass them onto ftpmaster instead of warning the
+      maintainer.
+    o need to do proper rfc822 escaping of from lines (as opposed to s/\.//g)
+    o Revisit linking of binary->source in install() in katie.
+    o Fix component handling in overrides (aj)
+    o Fix lack of entires in source overrides (aj)
+    o direport misreports things as section 'devel' (? we don't use direport)
+    o vrfy check of every Maintainer+Changed-By address; valid for 3 months.
+    o binary-all should be done on a per-source, per-architecture package
+      basis to avoid, e.g. the perl-modules problem.
+    o a source-missing-diff check: if the version has a - in it, and it
+      is sourceful, it needs orig and diff, e.g. if someone uploads
+      esound_0.2.22-6, and it is sourceful, and there is no diff ->
+      REJECT (version has a dash, therefore not debian native.)
+    o check linking of .tar.gz's to .dsc's.. see proftpd 1.2.1 as an example
+    o archive needs md5sum'ed regularly, but takes too long to do all
+      in one go; make progressive or weekly.
+    o katie/jenna/rhona/whatever needs to clear out .changes
+      files from p-u when removing stuff superseded by newer versions.
+      [but for now we have halle]
+    o test sig checking stuff in test/ (stupid thing is not modularized due to global abuse)
+    o when encountering suspicous things (e.g. file tainting) do something more drastic
+
+ * Easy:
+
+    o suite mapping and component mapping are parsed per changes file,
+      they should probably be stored in a dictionary created at startup.
+    o don't stat/md5sum files you have entries for in the DB, moron
+      boy (Katie.check_source_blah_blah)
+    o promote changes["changes"] to mandatory in katie.py(dump_vars)
+      after a month or so (or all .katie files contain in the queue
+      contain it).
+    o melanie should behave better with -a and without -b; see
+      gcc-defaults removal for an example.
+    o Reject on misconfigured kernel-package uploads
+    o utils.extract_component_from_section: main/utils -> main/utils, main rather than utils, main
+    o Fix jennier to warn if run when not in incoming or p-u
+    o katie should validate multi-suite uploads; only possible valid one
+      is "stable unstable"
+    o cron.daily* should change umask (aj sucks)
+    o Rene doesn't look at debian-installer but should.
+    o Rene needs to check for binary-less source packages.
+    o Rene could accept a suite argument (?)
+    o byhand stuff should send notification
+    o catherine should udpate db; move files, not the other way around [neuro]
+    o melanie should update the stable changelog [joey]
+    o update tagdb.dia
+  
+ * Bizzare/uncertain:
+   
+    o drop rather dubious currval stuff (?)
+    o rationalize os.path.join() usage
+    o Rene also doesn't seem to warn about missing binary packages (??)
+    o logging: hostname + pid ?
+    o ANAIS should be done in katie (?)
+    o Add an 'add' ability to melanie (? separate prog maybe)
+    o Replicate old dinstall report stuff (? needed ?)
+    o Handle the case of 1:1.1 which would overwrite 1.1 (?)
+    o maybe drop -r/--regex in madison, make it the default and
+      implement -e/--exact (a la joey's "elmo")
+    o dsc files are not checked for existence/perms (only an issue if
+      they're in the .dsc, but not the .changes.. possible?)
+  
+ * Cleanups & misc: 
+   
+    o db_access' get_files needs to use exceptions not this None, > 0, < 0 return val BS (?)
+    o The untouchable flag doesn't stop new packages being added to ``untouchable'' suites
+
+================================================================================
+
+Packaging
+---------
+
+  o Fix stuff to look in sensible places for libs and config file in debian package (?)
+
+================================================================================
+
+           --help      manpage
+-----------------------------------------------------------------------------
+alyson	      X
+amber	      X
+andrea	      X
+ashley	      X
+catherine     X		 X
+charisma      X		 X
+cindy	      X		 X
+claire	      X
+denise	      X
+fernanda      X
+halle	      X
+heidi	      X		 X
+helena        X
+jenna	      X
+jennifer      X
+jeri	      X
+julia	      X		 X
+kelly	      X		 X
+lisa	      X		 X
+madison	      X		 X
+melanie	      X		 X
+natalie	      X		 X
+neve	      X
+rene	      X
+rose	      X
+rhona	      X		 X
+saffron	      X
+shania	      X
+tea	      X
+ziyi	      X