queue_install: Adapt to new code organisation

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

Copy parts of process-unchecked to daklib.queue_install

To not bloat the new process-upload script unnecessarily

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

process-upload: clean up process_it()

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

process-upload: import process_it() from process-unchecked

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

process-upload: reorganise option parsing

Give more helpful error messages and log where we did get
the changes files from.

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

process-upload: Change logging behaviour for --no-action

Log to stderr instead if not at all.

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

Merge commit 'private/master' into process-upload

process-upload: Add standard boilerplate code

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

Merge branch 'merge'

* merge:
  Revert "remove an unneeded linewrap" - not needed anymore.
  Strip trailing newline, just like getstatusoutput

Merge commit 'origin/master'

Merge commit 'lamby/master' into merge

* commit 'lamby/master':
  Revert "remove an unneeded linewrap" - not needed anymore.
  Strip trailing newline, just like getstatusoutput

Signed-off-by: Joerg Jaspert <joerg@debian.org>

convert to using the debversion C type

Signed-off-by: Mark Hymers <mhy@debian.org>

Revert "remove an unneeded linewrap" - not needed anymore.

This reverts commit fa8582a41855d39d864495f1bc0b28805d992ad1.

Signed-off-by: Chris Lamb <lamby@debian.org>

Strip trailing newline, just like getstatusoutput

Signed-off-by: Chris Lamb <lamby@debian.org>

add missing s

Signed-off-by: Joerg Jaspert <joerg@debian.org>

remove an unneeded linewrap

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Misses summary

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Add suites to bug close mails from rm

Signed-off-by: Joerg Jaspert <joerg@debian.org>

dak rm

also print the bug closing reason within the bug close mail

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Document the current workings of p-u and p-a

So that we can find a way to merge them in one script.

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

you are named p-u, not dinstall

Signed-off-by: Joerg Jaspert <joerg@debian.org>

template

modify our contents template

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Oh well

Signed-off-by: Joerg Jaspert <joerg@debian.org>

And now this is clearly pythons fault. Ruby would have broken earlier! :)

Signed-off-by: Joerg Jaspert <joerg@debian.org>

And how about we make this mhy fault?

Signed-off-by: Joerg Jaspert <joerg@debian.org>

And that is probaby Linux (hello git) fault :)

Signed-off-by: Joerg Jaspert <joerg@debian.org>

And that is probably (not) ganneffs fault

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Its (not) probably all stews fault

Signed-off-by: Joerg Jaspert <joerg@debian.org>

knwonchnges

do not import subdirs for now.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge commit 'stew/knownchanges' into merge

* commit 'stew/knownchanges':
  trailing whitespace, update to 18 not 20
  update20 -> update 18
  removing known_changes now works, and inserting has a better chance of working
  update20 is closer to tested
  check for a DAK_CONFIG env variable when looking for dak.conf
  moved inserts of known_changes to Changes() class. add insert known_changes in p-u, remove known_changes in process_new.reject
  Start the known_changes table

Signed-off-by: Joerg Jaspert <joerg@debian.org>

trailing whitespace, update to 18 not 20

Signed-off-by: Mike O'Connor <stew@vireo.org>

merge with ftp-master

Signed-off-by: Mike O'Connor <stew@vireo.org>

update20 -> update 18

Signed-off-by: Mike O'Connor <stew@vireo.org>

Merge commit 'stew/contents' into merge

* commit 'stew/contents':
  bootstrap_bin working
  merge from master with sqla

Signed-off-by: Joerg Jaspert <joerg@debian.org>

merge from master

Signed-off-by: Mike O'Connor <stew@vireo.org>

Merge commit 'mhy/master' into merge

* commit 'mhy/master':
  update import_keyring
  now we don't need the dm flag
  allow for keyring priorities
  implement key acls
  add keyring defaults
  tidy up and add more methods to SQLA objects
  add basic mapper classes for new tables
  add acls to database schema
  fix indentation
  move check_status to daklib/queue.py
  move recheck logic into daklib/queue.py
  remove unused code (moving to process-holding) and readd security support

Signed-off-by: Joerg Jaspert <joerg@debian.org>

removing known_changes now works, and inserting has a better chance of working

Signed-off-by: Mike O'Connor <stew@dhcp-101.dfw1.kickstart.lan>

Merge commit 'lamby/master' into merge

* commit 'lamby/master':
  Correct local variable name in Upload.package_info.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Correct local variable name in Upload.package_info.

Signed-off-by: Chris Lamb <lamby@debian.org>

update20 is closer to tested

Signed-off-by: Mike O'Connor <stew@dhcp-101.dfw1.kickstart.lan>

check for a DAK_CONFIG env variable when looking for dak.conf

Signed-off-by: Mike O'Connor <stew@dhcp-101.dfw1.kickstart.lan>

Merge commit 'lamby/master' into merge

* commit 'lamby/master':
  Refactor Upload.package_info.
  Prefer interpolation over concatenation here.
  Add more spacing for reject messages/notes etc.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Refactor Upload.package_info.

Signed-off-by: Chris Lamb <lamby@debian.org>

moved inserts of known_changes to Changes() class. add insert known_changes in p-u, remove known_changes in process_new.reject

Signed-off-by: Mike O'Connor <stew@vireo.org>

Prefer interpolation over concatenation here.

Signed-off-by: Chris Lamb <lamby@debian.org>

Merge commit 'lamby/master' into merge

* commit 'lamby/master':
  Log whether maintainer can override when we lintian auto-reject.
  This is not icanhascheezburger.com :(

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Add more spacing for reject messages/notes etc.

Signed-off-by: Chris Lamb <lamby@debian.org>

Log whether maintainer can override when we lintian auto-reject.

Signed-off-by: Chris Lamb <lamby@debian.org>

This is not icanhascheezburger.com :(

Signed-off-by: Chris Lamb <lamby@debian.org>

Merge commit 'lamby/master' into merge

* commit 'lamby/master':
  Don't reject binary uploads. Sorry buildd maintainers.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Don't reject binary uploads. Sorry buildd maintainers.

Signed-off-by: Chris Lamb <lamby@debian.org>

Merge commit 'lamby/master' into merge

* commit 'lamby/master':
  Add NVIT test.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Add NVIT test.

Signed-off-by: Chris Lamb <lamby@debian.org>

Merge commit 'ftpmaster/master'

update import_keyring

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge commit 'lamby/master' into merge

* commit 'lamby/master':
  Fix another list.join/str.join instance.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Fix another list.join/str.join instance.

Signed-off-by: Chris Lamb <lamby@debian.org>

Merge commit 'lamby/master' into merge

* commit 'lamby/master':
  Prefer interpolation over concatenation.
  Join is str.join, not list.join

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Prefer interpolation over concatenation.

Signed-off-by: Chris Lamb <lamby@debian.org>

Join is str.join, not list.join

Signed-off-by: Chris Lamb <lamby@debian.org>

Merge commit 'lamby/master' into merge

* commit 'lamby/master':
  cnf.__getitem__ is not following protocol, but this is an easier fix.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

cnf.__getitem__ is not following protocol, but this is an easier fix.

Signed-off-by: Chris Lamb <lamby@debian.org>

Merge commit 'lamby/master' into merge

* commit 'lamby/master':
  We renamed s/directory/queue/ in this code.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

We renamed s/directory/queue/ in this code.

Signed-off-by: Chris Lamb <lamby@debian.org>

add some tags

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Start the known_changes table

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge branch 'merge'

* merge:
  Some people can't open the log; handle this gracefully.
  Split out symlink creation into Upload.ensure_orig

Merge commit 'lamby/master' into merge

* commit 'lamby/master':
  Some people can't open the log; handle this gracefully.
  Split out symlink creation into Upload.ensure_orig

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Some people can't open the log; handle this gracefully.

Signed-off-by: Chris Lamb <lamby@debian.org>

now we don't need the dm flag

Signed-off-by: Mark Hymers <mhy@debian.org>

allow for keyring priorities

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge commit 'ftpmaster/master'

implement key acls

Signed-off-by: Mark Hymers <mhy@debian.org>

Split out symlink creation into Upload.ensure_orig

Signed-off-by: Chris Lamb <lamby@debian.org>

Merge branch 'merge'

* merge:
  Check that we can parse the YAML file before we symlink anything.
  Test that we have Dinstall::LintianTags before we symlink anything
  Cleanup any symlinks that we created.

Merge commit 'lamby/master' into merge

* commit 'lamby/master':
  Check that we can parse the YAML file before we symlink anything.
  Test that we have Dinstall::LintianTags before we symlink anything
  Cleanup any symlinks that we created.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Check that we can parse the YAML file before we symlink anything.

Signed-off-by: Chris Lamb <lamby@debian.org>

Test that we have Dinstall::LintianTags before we symlink anything

Signed-off-by: Chris Lamb <lamby@debian.org>

Cleanup any symlinks that we created.

Signed-off-by: Chris Lamb <lamby@debian.org>

Merge branch 'merge'

* merge:
  Use the same session when getting pool entries, otherwise we can't do poolfile.location
  Hide SystemError, otherwise logging gets ugly.
  Don't reject orig tarballs - we want them

Merge commit 'lamby/master' into merge

* commit 'lamby/master':
  Use the same session when getting pool entries, otherwise we can't do poolfile.location
  Hide SystemError, otherwise logging gets ugly.
  Don't reject orig tarballs - we want them

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Use the same session when getting pool entries, otherwise we can't do poolfile.location

Signed-off-by: Chris Lamb <lamby@debian.org>

Hide SystemError, otherwise logging gets ugly.

Signed-off-by: Chris Lamb <lamby@debian.org>

Don't reject orig tarballs - we want them

Signed-off-by: Chris Lamb <lamby@debian.org>

Merge branch 'merge'

* merge:
  Don't ignore KeyboardInterrupt either in process_unchecked
  Don't log "dak top-level starting" every time we run "dak ls" or something.

Merge commit 'lamby/master' into merge

* commit 'lamby/master':
  Don't ignore KeyboardInterrupt either in process_unchecked
  Don't log "dak top-level starting" every time we run "dak ls" or something.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Don't ignore KeyboardInterrupt either in process_unchecked

Signed-off-by: Chris Lamb <lamby@debian.org>

Don't log "dak top-level starting" every time we run "dak ls" or something.

Signed-off-by: Chris Lamb <lamby@debian.org>

Merge branch 'merge'

* merge:
  Catch and log all exceptions from top-level dak command.
  Look in the pool and queues for missing orig files.
  Tidy check_lintian's 'log' closure
  Inline ensure_all_source_exists.

Merge commit 'lamby/master' into merge

* commit 'lamby/master':
  Catch and log all exceptions from top-level dak command.
  Look in the pool and queues for missing orig files.
  Tidy check_lintian's 'log' closure
  Inline ensure_all_source_exists.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Catch and log all exceptions from top-level dak command.

Signed-off-by: Chris Lamb <lamby@debian.org>

Look in the pool and queues for missing orig files.

Signed-off-by: Chris Lamb <lamby@debian.org>

Tidy check_lintian's 'log' closure

Thanks Enrico for calling it out.

Signed-off-by: Chris Lamb <lamby@debian.org>

dinstall order change

Signed-off-by: Joerg Jaspert <joerg@debian.org>

symlinks

no longer call symlinks in every dinstall run. instead call it once per
week.
Also call it in di-byhand, just to ensure there isnt any symlink going wrong
there.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Inline ensure_all_source_exists.

Signed-off-by: Chris Lamb <lamby@debian.org>

rsyncable

use --rsyncable for most of our gzip compressions.

For Packages.gz ... 2% bigger, 1/8th of the transfer
For Contents-lala.gz ... ~4% bigger, 1/4th of the transfer

Signed-off-by: Joerg Jaspert <joerg@debian.org>

quote

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge commit 'lamby/master' into merge

* commit 'lamby/master':
  Revert "While we're at it, split out another version of ensure_orig_files."

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Revert "While we're at it, split out another version of ensure_orig_files."

This reverts commit a51c070b5347ed9244c7b1db48bc4adb4801d24e.