-TODO:
--=-=-
+ TODO
+ ====
- o DONE
+[NB: I use this as a thought record/scribble, not everything on here
+ makes sense and/or is actually ever going to get done, so IIWY I
+ wouldn't use it as gospel for the future of katie or as a TODO
+ list for random hacking.]
-[Final stuff]
-------------
+================================================================================
-[Can be done later]
--------------------
+approved
+--------
- o when encountering suspicous things (e.g. file tainting) do something more drastic
+ o What to do with multi-suite uploads? Presumably hold in unapproved
+ and warn? Or what? Can't accept just for unstable or reject just
+ from stable.
- o test the REJECT paranoia stuff (katie.force_move())
+ o Whenever we check for anything in accepted we also need to check in
+ unapproved.
- o Go through each check validate in new setup
+ o non-sourceful uploads should go straight through if they have
+ source in accepted or the archive.
- o still not good crash-handling behaviour, too often end up with
- bogus rejects afterwards...
+ o security uploads on auric should be pre-approved.
- o stable installs are FUBAR
- o update tagdb.dia
+Others
+------
-==========================================================================
+[minor] kelly's copychanges, copykatie handling sucks, the per-suite
+ thing is static for all packages, so work out in advance dummy.
+[madison] # filenames ?
+[madison] # maintainer, component, install date (source only?), fingerprint?
-To fix
-======
+ o UrgencyLog stuff should minimize it's bombing out(?)
+ o Log stuff should open the log file
-More Urgent
------------
+ o [melanie] bomb out if a) no '[rene]' in the -m/--message, and b)
+ no other -d/--done or -C/--carbon-copy
+
+ o helena should footnote the actual notes, and also * the versions
+ with notes so we can see new versions since being noted...
+
+ o helena should have alternative sorting options, including reverse
+ and without or without differentiaion.
+
+ o julia should sync debadmin and ftpmaster (?)
+
+ o <drow> Can't read file.:
+ /org/security.debian.org/queue/accepted/accepted/apache-perl_1.3.9-14.1-1.21.20000309-1_sparc.katie.
+ You assume that the filenames are relative to accepted/, might want
+ to doc or fix that.
+
+ o <neuro> the orig was in NEW, the changes that caused it to be NEW
+ were pulled out in -2, and we end up with no orig in the archive
+ :(
+
+ o SecurityAcceptedAutoBuild doesn't handle the case of foo_3.3woody1
+ with a new .orig.tar.gz followed by a foo_3.3potato1 with the same
+ .orig.tar.gz; jennifer sees it and copes, but the AA code doesn't
+ and can't really easily know so the potato AA dir is left with no
+ .orig.tar.gz copy. doh.
+
+ o orig.tar.gz in accepted not handled properly (?)
+
+ o amber doesn't include .orig.tar.gz but it should
+
+ o permissions (paranoia, group write, etc.) configurability and overhaul
+
+ o remember duplicate copyrights in lisaand skip them, per package
+
+ o <M>ove option for lisa byhand proecessing
+
+ o rene could do with overrides
+
+ o db_access.get_location_id should handle the lack of archive_id properly
+
+ o the whole versioncmp thing should be documented
+
+ o lisa doesn't do the right thing with -2 and -1 uploads, as you can
+ end up with the .orig.tar.gz not in the pool
+
+ o lisa exits if you check twice (aj)
+
+ o lisa doesn't trap signals from fernanda properly
+
+ o queued and/or perl on sparc stable sucks - reimplement it.
+
+ o aj's bin nmu changes
+
+ o Lisa:
+ * priority >> optional
+ * arch != {any,all}
+ * build-depends wrong (via andrea)
+ * suid
+ * conficlits
+ * notification/stats to admin daily
+ o trap fernanda exiting
+ o distinguish binary only versus others (neuro)
+
+ o cache changes parsed from ordering (careful tho: would be caching
+ .changes from world writable incoming, not holding)
+
+ o katie doesn't recognise binonlyNMUs correctly in terms of telling
+ who their source is; source-must-exist does, but the info is not
+ propogated down.
o Fix BTS vs. katie sync issues by queueing(via BSMTP) BTS mail so
that it can be released on deman (e.g. ETRN to exim).
table. Then fix charisma to use them and write some scripting
to handle the Santiago situation. ]
- o (Depends) <aj> elmo: it should match \(\s*(<<|<|<=|=|>=|>|>>)\s*<VERSIONREGEXP>\) fwiw
+ o Validate Depends (et al.) [it should match \(\s*(<<|<|<=|=|>=|>|>>)\s*<VERSIONREGEXP>\)]
+
+ o Clean up DONE; archive to tar file every 2 weeks, update tar tvzf INDEX file.
+
+ o testing-updates suite: if binary-only and version << version in
+ unstable and source-ver ~= source-ver in testing; then map
+ unstable -> testing-updates ?
+
+ o hooks or configurability for debian specific checks (e.g. check_urgency, auto-building support)
+
+ o morgue needs auto-cleaning (?)
+
+================================================================================
Less Urgent
-----------
- * Harder:
+ o Accept "," as a seperator in -a, -s etc. args
- o reverse of source-must-exist; i.e. binary-for-source-must-not-exist
- o fernanda should check for conflicts and warn about them visavis priority [rmurray]
- o REJECT reminders in shania.
- o Clean up DONE; archive to tar file every 2 weeks, update tar tvzf INDEX file.
+ o change utils.copy to try rename() first
+
+ o [hard, long term] unchecked -> accepted should go into the db, not
+ a suite, but similar. this would allow katie to get even faster,
+ make madison more useful, decomplexify specialacceptedautobuild
+ and generally be more sane. may even be helpful to have e.g. new
+ in the DB, so that we avoid corner cases like the .orig.tar.gz
+ disappearing 'cos the package has been entirely removed but was
+ still on stayofexecution when it entered new.
+
+ o Logging [mostly done] (todo: rhona (hard), .. ?)
+
+ o jennifer is leaking file descriptors..
+
+ o the tar extractor class doesn't need to be redone for each package
+
+ o reverse of source-must-exist; i.e. binary-for-source-must-not-exist
+ o REJECT reminders in shania.
+ o fernanda should check for conflicts and warn about them visavis priority [rmurray]
+ o store a list of removed/files versions; also compare against them.
+ [but be careful about scalability]
+
+ o fernanda: print_copyright should be a lot more intelligent
+ @ handle copyright.gz
+ @ handle copyright.ja and copyright
+ @ handle (detect at least) symlinks to another package's doc directory
+ @ handle and/or fall back on source files (?)
+
+ o To incorporate from utils:
+ @ unreject
+
+ o auto-purge out-of-date stuff from non-free/contrib so that testing and stuff works
+ o doogie's binary -> source index
+ o jt's web stuff, matt's changelog stuff (overlap)
+
+ o [Hard] Need to merge non-non-US and non-US DBs.
+
+ o experimental needs to auto clean (relative to unstable) [partial: rene warns about this]
+
+ o Do a checkpc(1)-a-like which sanitizes a config files.
+ o fix parse_changes()/build_file_list() to sanity check filenames
+ o saftey check and/or rename debs so they match what they should be
+
+ o Improve andrea.
+ o Need to optimize all the queries by using EXAMINE and building some INDEXs.
+ [postgresql 7.2 will help here]
+ o Need to enclose all the setting SQL stuff in transactions (mostly done).
+ o Need to finish alyson (a way to sync katie.conf and the DB)
+ o Need the ability to rebuild all other tables from dists _or_ pools (in the event of disaster) (?)
+ o Make the --help and --version options do stuff for all scripts
+
+ o charisma can't handle whitespace-only lines (for the moment, this is feature)
+ o generic way of saying isabinary and isadsc. (?)
+
+ o s/distribution/suite/g
+
+ o cron.weekly:
+ @ weekly postins to d-c (?)
+ @ backup of report (?)
+ @ backup of changes.tgz (?)
+
+ * Harder:
+
+ o interrupting of stracing jennifer causes exceptions errors from apt_inst calls
+ o dependency checking (esp. stable) (partially done)
o override checks sucks; it needs to track changes made by the
maintainer and pass them onto ftpmaster instead of warning the
maintainer.
o need to do proper rfc822 escaping of from lines (as opposed to s/\.//g)
- o when dinstall is run in install mode but not as a cron job, it
- should probably run jenna to avoid madison originated user confusion
o Revisit linking of binary->source in install() in katie.
o Fix component handling in overrides (aj)
o Fix lack of entires in source overrides (aj)
- o direport misreports things as section 'devel'
+ o direport misreports things as section 'devel' (? we don't use direport)
o vrfy check of every Maintainer+Changed-By address; valid for 3 months.
o binary-all should be done on a per-source, per-architecture package
basis to avoid, e.g. the perl-modules problem.
o katie/jenna/rhona/whatever needs to clear out .changes
files from p-u when removing stuff superseded by newer versions.
[but for now we have halle]
- o test sig checking stuff in test/ (stupid thing is not modularized)
+ o test sig checking stuff in test/ (stupid thing is not modularized due to global abuse)
+ o when encountering suspicous things (e.g. file tainting) do something more drastic
* Easy:
- o Rationalize config stuff.. Dir::Foo not Dir::FooDir [breaks other people??]
+ o suite mapping and component mapping are parsed per changes file,
+ they should probably be stored in a dictionary created at startup.
+ o don't stat/md5sum files you have entries for in the DB, moron
+ boy (Katie.check_source_blah_blah)
+ o promote changes["changes"] to mandatory in katie.py(dump_vars)
+ after a month or so (or all .katie files contain in the queue
+ contain it).
o melanie should behave better with -a and without -b; see
gcc-defaults removal for an example.
o Reject on misconfigured kernel-package uploads
- o denise abuses sys.stdout == badly
o utils.extract_component_from_section: main/utils -> main/utils, main rather than utils, main
- o Fix katie to warn if run when not in incoming or p-u
- o check for empty debs; check for empty (or small (<1k ?)) .tar.gz's.
+ o Fix jennier to warn if run when not in incoming or p-u
o katie should validate multi-suite uploads; only possible valid one
is "stable unstable"
o cron.daily* should change umask (aj sucks)
o Rene doesn't look at debian-installer but should.
o Rene needs to check for binary-less source packages.
o Rene could accept a suite argument (?)
- o run shania every day (?) [ryan]
- o we don't check for .orig.tar.gz's, so it's possible to upload -2 with no .orig.tar.gz.
o byhand stuff should send notification
o catherine should udpate db; move files, not the other way around [neuro]
o melanie should update the stable changelog [joey]
+ o update tagdb.dia
* Bizzare/uncertain:
- o validate source consistency, i.e. .dsc and .tar.gz as minimum (??)
+ o drop rather dubious currval stuff (?)
+ o rationalize os.path.join() usage
o Rene also doesn't seem to warn about missing binary packages (??)
- o jenna should not delete things because they don't exist (?)
+ o logging: hostname + pid ?
+ o ANAIS should be done in katie (?)
+ o Add an 'add' ability to melanie (? separate prog maybe)
+ o Replicate old dinstall report stuff (? needed ?)
+ o Handle the case of 1:1.1 which would overwrite 1.1 (?)
+ o maybe drop -r/--regex in madison, make it the default and
+ implement -e/--exact (a la joey's "elmo")
+ o dsc files are not checked for existence/perms (only an issue if
+ they're in the .dsc, but not the .changes.. possible?)
* Cleanups & misc:
o db_access' get_files needs to use exceptions not this None, > 0, < 0 return val BS (?)
o The untouchable flag doesn't stop new packages being added to ``untouchable'' suites
- o jenna is too slow
- o jenna doesn't handle arch: any -> arch: all transitions
- [aj worked around; need to revisit]
-
- * Stable:
-
-<neuro> well, there's one other thing that doesn't work so well
-<neuro> you probably should have stable_install ignore Closes:
-<neuro> or it might be unique to the case of stable unstable uploads
-
-
-
-===================================================================================================
-TODO
-====
-
-More Urgent
------------
-
- o testing-updates suite: if binary-only and version << version in
- unstable and source-ver ~= source-ver in testing; then map
- unstable -> testing-updates ?
-
- o [Complexish] Secure incoming handling
- o [Hard] dependency checking (esp. stable) (partially done)
- o hooks for debian specific checks (e.g. check_urgency)
- o Logging [mostly done]
- o Abstraction of suite hardcoding (q.v. progengy)
- o morgue needs auto-cleaning
+================================================================================
Packaging
---------
- o Install python libraries (db_access and utils)
- o Install config file
o Fix stuff to look in sensible places for libs and config file in debian package (?)
- o man pages and/or documentation
-
-Less Urgent
------------
-
- o store a list of removed/files versions; also compare against them.
- be careful about scalability.
-
- o fernanda: print_copyright should be a lot more intelligent
- @ handle copyright.gz
- @ handle copyright.ja and copyright
- @ handle (detect at least) symlinks to another package's doc directory
- @ handle and/or fall back on source files?
- @ check only NEW packages out of a source package with some NEW, some old
- @ check latest version if given multiples!
- o To incorporate from utils:
- @ unreject
- @ genreport
- @ reject
- @ newchanges
-
- o logging: hostname + pid ?
-
- o katie could vrfy email addresses (?) [errge@]
- o auto-purge out-of-date stuff from non-free/contrib so that testing and stuff works
- o doogie's binary -> source index
- o ANAIS should be done in katie (?)
- o Add an 'add' ability to melanie (? separate prog maybe)
- o Add urgency + installed date to the DB for aj/testing (? still needed ?)
- o jt's web stuff, matt's changelog stuff (overlap)
-
- o [Hard] Need to merge non-non-US and non-US DBs.
-
- o experimental needs to auto clean (relative to unstable) [warns in rene about this]
-
- o Do a checkpc(1)-a-like which sanitizes a config files.
- o fix parse_changes()/build_file_list() to sanity check filenames
- o saftey check and/or rename debs so they match what they should be
-
- o Improve andrea.
- o Need to optimize all the queries by using EXAMINE and building some INDEXs.
- o Need to enclose all the setting SQL stuff in transactions (mostly done).
- o Need to finish alyson (a way to sync katie.conf and the DB)
- o Need the ability to rebuild all other tables from dists _or_ pools (in the event of disaster) (?)
- o Make the --help and --version options do stuff for all scripts
- o Need to check for .dsc when source is mentioned
-
- o charisma can't handle whitespace-only lines (for the moment, this is feature)
- o Should use $EDITOR, not hardcode vi
- o Should reject packages with bad timestamps
- o Replicate old dinstall report stuff (? needed ?)
- o Handle the case of 1:1.1 which would overwrite 1.1 (?)
-
- o generic way of saying isabinary and isadsc. (?)
-
- o s/distribution/suite/g
-
- o cron.weekly:
- @ weekly postins to d-c (?)
- @ backup of report (?)
- @ backup of changes.tgz (?)
+================================================================================