-TODO:
--=-=-
+ TODO
+ ====
- o DONE
+[NB: I use this as a thought record/scribble, not everything on here
+ makes sense and/or is actually ever going to get done, so IIWY I
+ wouldn't use it as gospel for the future of katie or as a TODO
+ list for random hacking.]
-[Final stuff]
-------------
+================================================================================
-[Can be done later]
--------------------
+Actually Urgent
+---------------
- o when encountering suspicous things (e.g. file tainting) do something more drastic
+ o UrgencyLog stuff should minimize it's bombing out(?)
+ o Log stuff should open the log file g+w lamer
+<aj> elmo_home: (although, if you haven't already, you might want to add a `Reject: invalid character in Version:' check to katie sometime)
+<aj> elmo_home: (at least while auric's apt doesn't handle ~'s)
- o test the REJECT paranoia stuff (katie.force_move())
+More Urgent
+-----------
- o Go through each check validate in new setup
+ <drow> Can't read file.: /org/security.debian.org/queue/accepted/accepted/apache-perl_1.3.9-14.1-1.21.20000309-1_sparc.katie. You assume that the filenames are relative to accepted/, might want to doc or fix that.
- o still not good crash-handling behaviour, too often end up with
- bogus rejects afterwards...
+ o permissions (paranoia, group write, etc.) configurability and overhaul
- o stable installs are FUBAR
+ o <M>ove option for lisa byhand proecessing
- o update tagdb.dia
+ o rene could do with overrides
+
+ o db_access.get_location_id should handle the lack of archive_id properly
-==========================================================================
+ o the whole versioncmp thing should be documented
+ o lisa doesn't do the right thing with -2 and -1 uploads, as you can
+ end up with the .orig.tar.gz not in the pool
-To fix
-======
+ o lisa exits if you check twice (aj)
-More Urgent
------------
+ o lisa doesn't trap signals from fernanda properly
+
+ o queued and/or perl on sparc stable sucks - reimplement it.
+
+ o aj's bin nmu changes
+
+ o Lisa:
+ * priority >> optional
+ * arch != {any,all}
+ * build-depends wrong (via andrea)
+ * suid
+ * conficlits
+ * notification/stats to admin daily
+ o trap fernanda exiting
+ o distinguish binary only versus others (neuro)
+
+ o cache changes parsed from ordering (careful tho: would be caching
+ .changes from world writable incoming, not holding); katie should
+ certainly be using .katie files, not parsing changes, that's just
+ silly.
+
+ o katie doesn't recognise binonlyNMUs correctly in terms of telling
+ who their source is; source-must-exist does, but the info is not
+ propogated down.
o Fix BTS vs. katie sync issues by queueing(via BSMTP) BTS mail so
that it can be released on deman (e.g. ETRN to exim).
o (Depends) <aj> elmo: it should match \(\s*(<<|<|<=|=|>=|>|>>)\s*<VERSIONREGEXP>\) fwiw
+ o Clean up DONE; archive to tar file every 2 weeks, update tar tvzf INDEX file.
+
+ o testing-updates suite: if binary-only and version << version in
+ unstable and source-ver ~= source-ver in testing; then map
+ unstable -> testing-updates ?
+
+ o hooks or configurability for debian specific checks (e.g. check_urgency, auto-building support)
+
+ o morgue needs auto-cleaning (?)
+
+================================================================================
+
Less Urgent
-----------
- * Harder:
+ o change utils.copy to try rename() first
+
+ o [hard, long term] unchecked -> accepted should go into the db, not
+ a suite, but similar. this would allow katie to get even faster,
+ make madison more useful, decomplexify specialacceptedautobuild and
+ generally be more sane. may even be helpful to have e.g. new in the
+ DB, so that we avoid corner cases like the .orig.tar.gz disappearing
+ 'cos the package has been entirely removed but was still on
+ stayofexecution when it entered new.
+
+ o Logging [mostly done] (todo: rhona (hard), .. ?)
+
+ o jennifer is leaking file descriptors..
+
+ o the tar extractor class doesn't need to be redone for each package
+
+ o reverse of source-must-exist; i.e. binary-for-source-must-not-exist
+ o REJECT reminders in shania.
+ o fernanda should check for conflicts and warn about them visavis priority [rmurray]
+ o store a list of removed/files versions; also compare against them.
+ [but be careful about scalability]
+
+ o fernanda: print_copyright should be a lot more intelligent
+ @ handle copyright.gz
+ @ handle copyright.ja and copyright
+ @ handle (detect at least) symlinks to another package's doc directory
+ @ handle and/or fall back on source files (?)
+
+ o To incorporate from utils:
+ @ unreject
+
+ o auto-purge out-of-date stuff from non-free/contrib so that testing and stuff works
+ o doogie's binary -> source index
+ o jt's web stuff, matt's changelog stuff (overlap)
+
+ o [Hard] Need to merge non-non-US and non-US DBs.
+
+ o experimental needs to auto clean (relative to unstable) [partial: rene warns about this]
+
+ o Do a checkpc(1)-a-like which sanitizes a config files.
+ o fix parse_changes()/build_file_list() to sanity check filenames
+ o saftey check and/or rename debs so they match what they should be
+
+ o Improve andrea.
+ o Need to optimize all the queries by using EXAMINE and building some INDEXs.
+ [postgresql 7.2 will help here]
+ o Need to enclose all the setting SQL stuff in transactions (mostly done).
+ o Need to finish alyson (a way to sync katie.conf and the DB)
+ o Need the ability to rebuild all other tables from dists _or_ pools (in the event of disaster) (?)
+ o Make the --help and --version options do stuff for all scripts
+
+ o charisma can't handle whitespace-only lines (for the moment, this is feature)
+
+ o generic way of saying isabinary and isadsc. (?)
+
+ o s/distribution/suite/g
+
+ o cron.weekly:
+ @ weekly postins to d-c (?)
+ @ backup of report (?)
+ @ backup of changes.tgz (?)
- o reverse of source-must-exist; i.e. binary-for-source-must-not-exist
- o fernanda should check for conflicts and warn about them visavis priority [rmurray]
- o REJECT reminders in shania.
- o Clean up DONE; archive to tar file every 2 weeks, update tar tvzf INDEX file.
+ * Harder:
+ o interrupting of stracing jennifer causes exceptions errors from apt_inst calls
+ o dependency checking (esp. stable) (partially done)
o override checks sucks; it needs to track changes made by the
maintainer and pass them onto ftpmaster instead of warning the
maintainer.
o need to do proper rfc822 escaping of from lines (as opposed to s/\.//g)
- o when dinstall is run in install mode but not as a cron job, it
- should probably run jenna to avoid madison originated user confusion
o Revisit linking of binary->source in install() in katie.
o Fix component handling in overrides (aj)
o Fix lack of entires in source overrides (aj)
- o direport misreports things as section 'devel'
+ o direport misreports things as section 'devel' (? we don't use direport)
o vrfy check of every Maintainer+Changed-By address; valid for 3 months.
o binary-all should be done on a per-source, per-architecture package
basis to avoid, e.g. the perl-modules problem.
o katie/jenna/rhona/whatever needs to clear out .changes
files from p-u when removing stuff superseded by newer versions.
[but for now we have halle]
- o test sig checking stuff in test/ (stupid thing is not modularized)
+ o test sig checking stuff in test/ (stupid thing is not modularized due to global abuse)
+ o when encountering suspicous things (e.g. file tainting) do something more drastic
* Easy:
- o Rationalize config stuff.. Dir::Foo not Dir::FooDir [breaks other people??]
+ o suite mapping and component mapping are parsed per changes file,
+ they should probably be stored in a dictionary created at startup.
+ o don't stat/md5sum files you have entries for in the DB, moron
+ boy (Katie.check_source_blah_blah)
+ o promote changes["changes"] to mandatory in katie.py(dump_vars)
+ after a month or so (or all .katie files contain in the queue
+ contain it).
o melanie should behave better with -a and without -b; see
gcc-defaults removal for an example.
o Reject on misconfigured kernel-package uploads
- o denise abuses sys.stdout == badly
o utils.extract_component_from_section: main/utils -> main/utils, main rather than utils, main
- o Fix katie to warn if run when not in incoming or p-u
- o check for empty debs; check for empty (or small (<1k ?)) .tar.gz's.
+ o Fix jennier to warn if run when not in incoming or p-u
o katie should validate multi-suite uploads; only possible valid one
is "stable unstable"
o cron.daily* should change umask (aj sucks)
o Rene doesn't look at debian-installer but should.
o Rene needs to check for binary-less source packages.
o Rene could accept a suite argument (?)
- o run shania every day (?) [ryan]
- o we don't check for .orig.tar.gz's, so it's possible to upload -2 with no .orig.tar.gz.
o byhand stuff should send notification
o catherine should udpate db; move files, not the other way around [neuro]
o melanie should update the stable changelog [joey]
+ o update tagdb.dia
* Bizzare/uncertain:
- o validate source consistency, i.e. .dsc and .tar.gz as minimum (??)
+ o drop rather dubious currval stuff (?)
+ o rationalize os.path.join() usage
o Rene also doesn't seem to warn about missing binary packages (??)
- o jenna should not delete things because they don't exist (?)
+ o logging: hostname + pid ?
+ o ANAIS should be done in katie (?)
+ o Add an 'add' ability to melanie (? separate prog maybe)
+ o Replicate old dinstall report stuff (? needed ?)
+ o Handle the case of 1:1.1 which would overwrite 1.1 (?)
+ o maybe drop -r/--regex in madison, make it the default and
+ implement -e/--exact (a la joey's "elmo")
* Cleanups & misc:
o db_access' get_files needs to use exceptions not this None, > 0, < 0 return val BS (?)
o The untouchable flag doesn't stop new packages being added to ``untouchable'' suites
- o jenna is too slow
- o jenna doesn't handle arch: any -> arch: all transitions
- [aj worked around; need to revisit]
-
- * Stable:
-
-<neuro> well, there's one other thing that doesn't work so well
-<neuro> you probably should have stable_install ignore Closes:
-<neuro> or it might be unique to the case of stable unstable uploads
-
-
-===================================================================================================
-
-TODO
-====
-
-More Urgent
------------
-
- o testing-updates suite: if binary-only and version << version in
- unstable and source-ver ~= source-ver in testing; then map
- unstable -> testing-updates ?
-
- o [Complexish] Secure incoming handling
- o [Hard] dependency checking (esp. stable) (partially done)
- o hooks for debian specific checks (e.g. check_urgency)
- o Logging [mostly done]
- o Abstraction of suite hardcoding (q.v. progengy)
- o morgue needs auto-cleaning
+================================================================================
Packaging
---------
- o Install python libraries (db_access and utils)
- o Install config file
o Fix stuff to look in sensible places for libs and config file in debian package (?)
- o man pages and/or documentation
-
-Less Urgent
------------
-
- o store a list of removed/files versions; also compare against them.
- be careful about scalability.
-
- o fernanda: print_copyright should be a lot more intelligent
- @ handle copyright.gz
- @ handle copyright.ja and copyright
- @ handle (detect at least) symlinks to another package's doc directory
- @ handle and/or fall back on source files?
- @ check only NEW packages out of a source package with some NEW, some old
- @ check latest version if given multiples!
-
- o To incorporate from utils:
- @ unreject
- @ genreport
- @ reject
- @ newchanges
-
- o logging: hostname + pid ?
- o katie could vrfy email addresses (?) [errge@]
- o auto-purge out-of-date stuff from non-free/contrib so that testing and stuff works
- o doogie's binary -> source index
- o ANAIS should be done in katie (?)
- o Add an 'add' ability to melanie (? separate prog maybe)
- o Add urgency + installed date to the DB for aj/testing (? still needed ?)
- o jt's web stuff, matt's changelog stuff (overlap)
-
- o [Hard] Need to merge non-non-US and non-US DBs.
-
- o experimental needs to auto clean (relative to unstable) [warns in rene about this]
-
- o Do a checkpc(1)-a-like which sanitizes a config files.
- o fix parse_changes()/build_file_list() to sanity check filenames
- o saftey check and/or rename debs so they match what they should be
-
- o Improve andrea.
- o Need to optimize all the queries by using EXAMINE and building some INDEXs.
- o Need to enclose all the setting SQL stuff in transactions (mostly done).
- o Need to finish alyson (a way to sync katie.conf and the DB)
- o Need the ability to rebuild all other tables from dists _or_ pools (in the event of disaster) (?)
- o Make the --help and --version options do stuff for all scripts
- o Need to check for .dsc when source is mentioned
-
- o charisma can't handle whitespace-only lines (for the moment, this is feature)
- o Should use $EDITOR, not hardcode vi
- o Should reject packages with bad timestamps
- o Replicate old dinstall report stuff (? needed ?)
- o Handle the case of 1:1.1 which would overwrite 1.1 (?)
-
- o generic way of saying isabinary and isadsc. (?)
-
- o s/distribution/suite/g
-
- o cron.weekly:
- @ weekly postins to d-c (?)
- @ backup of report (?)
- @ backup of changes.tgz (?)
+================================================================================