]> git.decadent.org.uk Git - dak.git/log
dak.git
8 years agodebian-security/dak.conf: set changelog export directory
Ansgar Burchardt [Mon, 16 May 2016 17:57:15 +0000 (19:57 +0200)]
debian-security/dak.conf: set changelog export directory

8 years agoMerge remote-tracking branch 'jcristau/by-hash-v4'
Ansgar Burchardt [Mon, 16 May 2016 15:58:50 +0000 (17:58 +0200)]
Merge remote-tracking branch 'jcristau/by-hash-v4'

8 years agoAdd by-hash support
Julien Cristau [Wed, 27 Apr 2016 08:06:08 +0000 (10:06 +0200)]
Add by-hash support

Add a per-suite boolean to enable by-hash; store the by-hash files in
the db, and record when they stopped being referenced, so that
clean-suites can delete them after the archive's stayofexecution time.

In generate-release, where we have checksums for all the things,
hardlink files to the by-hash dir for each of the suite's configured
hash methods.

Signed-off-by: Julien Cristau <jcristau@debian.org>
---
changes in v2:
- use archive.stayofexecution as delay before removing files from
  by-hash
- don't assume any particular ordering for suite.checksums

changes in v3:
- rebase on latest master, update115 is now update116
- handle missing files in clean_byhash

changes in v4:
- use hardlinks instead of symlinks
- don't initialize `unreferenced` to its default value
- in clean_byhash, remove useless fetchall, and handle ENOENT from
  unlink instead of checking for existence beforehand

8 years agoAdd python-debianbts as a dependency in setup/README (Closes: #823118)
Luke Faraone [Sat, 30 Apr 2016 21:59:52 +0000 (21:59 +0000)]
Add python-debianbts as a dependency in setup/README (Closes: #823118)

Needed to run the test suite:

    ======================================================================
    ERROR: test_importing_rm (test_imports.ImportTestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/srv/daksrc/tests/test_imports.py", line 15, in test_fn
        __import__('dak', fromlist=[cmd])
      File "/srv/daksrc/dak/rm.py", line 54, in <module>
        from daklib.rm import remove
      File "/srv/daksrc/dak/daklib/rm.py", line 52, in <module>
        import debianbts as bts
    ImportError: No module named debianbts

Signed-off-by: Luke Faraone <lfaraone@debian.org>
8 years agoNotify w-b earlier
Ansgar Burchardt [Tue, 3 May 2016 20:57:57 +0000 (22:57 +0200)]
Notify w-b earlier

Before Debian LTS started to use the security archive, all uploads that
would trigger buildds would be sourceful uploads to a policy queue. So
new packages entering the build queues and pushing the public mirrors
would likely not happen in the same cron.unchecked run.

With Debian LTS, uploads would go to both the build queues and trigger a
public mirror push. If the mirror push takes more than three minutes,
cron.buildd would however not notify w-b of changes.

This patch moves the w-b notification right behind the update of the
build queues and before the mirror push to avoid the problem. Ideally
we would not rely on anything happening within the last three minutes
though.

8 years agodak: don't ignore SystemExit
Julien Cristau [Wed, 27 Apr 2016 08:26:28 +0000 (10:26 +0200)]
dak: don't ignore SystemExit

Let the python interpreter handle it and actually exit with the right
status, otherwise when a sub-command calls sys.exit(1) we end up exiting
0 instead, which is confusing if not dangerous.

Signed-off-by: Julien Cristau <jcristau@debian.org>
8 years agoPublish security archive on all changes
Ansgar Burchardt [Sun, 1 May 2016 14:56:35 +0000 (16:56 +0200)]
Publish security archive on all changes

With "wheezy" being used for Debian LTS, the archive needs to be
published not only when policy queue uploads are accepted, but also
when direct uploads to "wheezy" arrive.

The same applies to uploads to "jessie-kfreebsd".

8 years agoKeep track of when a package was last added to a suite
Ansgar Burchardt [Sun, 1 May 2016 14:55:11 +0000 (16:55 +0200)]
Keep track of when a package was last added to a suite

8 years agoWork with newer SQLAlchemy versions
Ansgar Burchardt [Sun, 1 May 2016 14:48:22 +0000 (16:48 +0200)]
Work with newer SQLAlchemy versions

Newer SQLAlchemy versions seem to track the `binary` attribute of
`BinaryMetadata`.  This means the association proxy in `DBBinary`
would first create a `BinaryMetadata` with `binary` set to `None`,
flush this to the database and then set the actual value of `binary`
we want.  However the flush fails, as `NULL` is not allowed for the
`binary_id` column.

The problem can be avoided by not setting `binary` to `None`.  This is
treated as an "undefined value" by SQLAlchemy and changing it later
means the database row never sees `NULL`.

The same applies to the `source` attribute of `SourceMetadata`.

8 years agoMove more specific source file regexes before general ones
Guillem Jover [Sat, 23 Apr 2016 12:02:16 +0000 (14:02 +0200)]
Move more specific source file regexes before general ones

The orig tarball regexes match also the orig tarball detached
signatures because they are not anchored at the end, and the loop
short-circuits on first match. So we have to move them before their
more general counterparts.

Fixes commit 84f230b6cfb4b3eb1ef5fbe672e121b1a8e7aabb.

8 years agoRead files in chunks
Ansgar Burchardt [Fri, 15 Apr 2016 07:49:54 +0000 (09:49 +0200)]
Read files in chunks

Calling `f.read()` would read the entire file at once which is
wasteful. It's also not intended given the read loop.

Also replace a use of `readline()` with `read(65536)` as well where we
want to read the entire file and not only single lines.

8 years agoDo not pass all file descriptiors to external process.
Ansgar Burchardt [Fri, 15 Apr 2016 06:37:10 +0000 (08:37 +0200)]
Do not pass all file descriptiors to external process.

8 years agoFix support for detached upstream signatures
Guillem Jover [Thu, 11 Feb 2016 09:27:16 +0000 (10:27 +0100)]
Fix support for detached upstream signatures

We should only allow upstream tarballs with detached signatures, but not
native tarballs, «<source>_<version>.orig.asc» or «<source>_<version>.asc»
filenames. We also need to allow this on all relevant regexes. This commit
fixes and refactors the regexes to implement the above, and to make it
easier to modify a single place in the future.

Fixes commit 4911282cbf231b116872a6f16eb838f7862ebc36.

Closes: #759401
8 years agoStop using undefined reject function in check_dsc_files
Guillem Jover [Thu, 11 Feb 2016 09:27:13 +0000 (10:27 +0100)]
Stop using undefined reject function in check_dsc_files

When there's an unknown file in a source package, the function spits
this:

  ,---
  Processing raised an exception: global name 'reject' is not defined.
  Traceback (most recent call last):
    File "/srv/ftp-master.debian.org/dak/dak/daklib/archive.py", line 966, in check
      chk().check(self)
    File "/srv/ftp-master.debian.org/dak/dak/daklib/checks.py", line 497, in check
      rejects = utils.check_dsc_files(dsc_fn, control, source.files.keys())
    File "/srv/ftp-master.debian.org/dak/dak/daklib/utils.py", line 343, in check_dsc_files
      reject("%s: unexpected source file '%s'" % (dsc_filename, f))
  NameError: global name 'reject' is not defined
  `---

Just set the reject message and break from the loop.

8 years agoRemove unused re_source_ext import
Guillem Jover [Thu, 11 Feb 2016 09:27:10 +0000 (10:27 +0100)]
Remove unused re_source_ext import

8 years agoEscape dots in file type check in check_dsc_files
Guillem Jover [Thu, 11 Feb 2016 09:26:57 +0000 (10:26 +0100)]
Escape dots in file type check in check_dsc_files

8 years agoImplement mirrorpush-release function
Joerg Jaspert [Sat, 2 Apr 2016 13:03:52 +0000 (15:03 +0200)]
Implement mirrorpush-release function

8 years agoAdd "mirror" so POSSIBLEARGS
Ansgar Burchardt [Sat, 2 Apr 2016 12:21:02 +0000 (14:21 +0200)]
Add "mirror" so POSSIBLEARGS

8 years agoREADME.stable-point-release: updates from 7.10
Ansgar Burchardt [Sat, 2 Apr 2016 12:20:41 +0000 (14:20 +0200)]
README.stable-point-release: updates from 7.10

8 years agoUps, that backports dir is still in use
Joerg Jaspert [Sat, 2 Apr 2016 12:05:22 +0000 (14:05 +0200)]
Ups, that backports dir is still in use

8 years agoGenerate new.html stuff atomic
Joerg Jaspert [Sat, 2 Apr 2016 12:01:37 +0000 (14:01 +0200)]
Generate new.html stuff atomic

8 years agoRemove an old backports-master dir, remove lock unchecked after usage
Joerg Jaspert [Sat, 2 Apr 2016 11:59:34 +0000 (13:59 +0200)]
Remove an old backports-master dir, remove lock unchecked after usage

8 years agoREADME.stable-point-release: updates from 8.4
Ansgar Burchardt [Sat, 2 Apr 2016 10:05:04 +0000 (12:05 +0200)]
README.stable-point-release: updates from 8.4

8 years agoUse parameter passed to the --suite option
Ansgar Burchardt [Thu, 31 Mar 2016 19:17:44 +0000 (21:17 +0200)]
Use parameter passed to the --suite option

We used the remaining parameters before which is somewhat confusing.

8 years agoThe --suite option has an argument.
Ansgar Burchardt [Thu, 31 Mar 2016 19:12:54 +0000 (21:12 +0200)]
The --suite option has an argument.

8 years agodebian: Add release team key to import testing-debug
Ansgar Burchardt [Thu, 31 Mar 2016 18:32:50 +0000 (20:32 +0200)]
debian: Add release team key to import testing-debug

8 years agoRevert "Use only 'xz' as default compression for Sources and Packages"
Ansgar Burchardt [Wed, 30 Mar 2016 22:04:58 +0000 (00:04 +0200)]
Revert "Use only 'xz' as default compression for Sources and Packages"

This reverts commit 196b3433d455b925758d6a46751a1f0b01617213.

8 years agoAcquire a lock in the main process when writing Contents
Ansgar Burchardt [Wed, 30 Mar 2016 21:36:23 +0000 (23:36 +0200)]
Acquire a lock in the main process when writing Contents

This should prevent changes to the {src,bin}_contents tables and result
in all helper processes seeing the same data.

8 years agoUse only 'xz' as default compression for Sources and Packages
Ansgar Burchardt [Wed, 30 Mar 2016 21:26:17 +0000 (23:26 +0200)]
Use only 'xz' as default compression for Sources and Packages

8 years agoRemove files that are (no longer) generated
Ansgar Burchardt [Wed, 30 Mar 2016 21:23:13 +0000 (23:23 +0200)]
Remove files that are (no longer) generated

This makes it less error-prone to switch compression methods: the files
that are no longer generated are automatically removed. Before they had
to be removed manually.

8 years agoHopefully enable g-i-d to deal with .xz files
Joerg Jaspert [Tue, 15 Mar 2016 22:25:59 +0000 (23:25 +0100)]
Hopefully enable g-i-d to deal with .xz files

which is helpful when there are no .gz/.bz2 ones around to
look at.

8 years agoRemove suite alias, use suite directly
Joerg Jaspert [Tue, 15 Mar 2016 21:57:18 +0000 (22:57 +0100)]
Remove suite alias, use suite directly

8 years agoMerge remote-tracking branch 'juliank/master'
Joerg Jaspert [Tue, 15 Mar 2016 21:44:30 +0000 (22:44 +0100)]
Merge remote-tracking branch 'juliank/master'

* juliank/master:
  Only generate binary checksum fields for the requested ones
  Only generate fields in Sources for requested checksums

Signed-off-by: Joerg Jaspert <joerg@debian.org>
8 years agoReplace hardcoded hashfuncs with values generated from database
Joerg Jaspert [Sun, 13 Mar 2016 14:23:34 +0000 (15:23 +0100)]
Replace hardcoded hashfuncs with values generated from database

8 years agoOnly generate binary checksum fields for the requested ones
Julian Andres Klode [Sun, 13 Mar 2016 14:01:46 +0000 (15:01 +0100)]
Only generate binary checksum fields for the requested ones

This is the second step. As we are simply concatenating things,
we can just concatenate an empty string if we do not want a
checksum.

8 years agoOnly generate fields in Sources for requested checksums
Julian Andres Klode [Sun, 13 Mar 2016 13:48:51 +0000 (14:48 +0100)]
Only generate fields in Sources for requested checksums

We make use of the fact that STRING_AGG ignores NULL values
when joining the values, so we do not get empty lines in there.

8 years agoAdd checksums column to suite table
Joerg Jaspert [Sun, 13 Mar 2016 13:29:54 +0000 (14:29 +0100)]
Add checksums column to suite table

8 years agoNo need to export backports changelogs anymore
Joerg Jaspert [Sun, 13 Mar 2016 13:02:41 +0000 (14:02 +0100)]
No need to export backports changelogs anymore

8 years agoStop work in the backports archive
Joerg Jaspert [Sun, 13 Mar 2016 12:57:40 +0000 (13:57 +0100)]
Stop work in the backports archive

Mind, its still configured and exists, need to remove that, but this
should stop pushing the mirrors and changing it around.

8 years agoArchitecture names can have more than one character
Ansgar Burchardt [Sat, 6 Feb 2016 13:38:58 +0000 (14:38 +0100)]
Architecture names can have more than one character

8 years agoComponents-*.yml can exist for all architectures
Ansgar Burchardt [Sat, 6 Feb 2016 11:37:00 +0000 (12:37 +0100)]
Components-*.yml can exist for all architectures

8 years agoTake away static-update-components stdin, its ssh eats it
Joerg Jaspert [Sat, 23 Jan 2016 21:33:53 +0000 (22:33 +0100)]
Take away static-update-components stdin, its ssh eats it

Which makes running it inside a while loop a pain.
(Its ssh could use a -n,but this works too)

8 years agoVarious small fixups/style issues
Joerg Jaspert [Sat, 23 Jan 2016 21:17:15 +0000 (22:17 +0100)]
Various small fixups/style issues

8 years agoREADME.stable-point-release: updates from 8.3
Ansgar Burchardt [Sat, 23 Jan 2016 13:38:35 +0000 (14:38 +0100)]
README.stable-point-release: updates from 8.3

8 years agoAdd new "mirror" cronscript
Ansgar Burchardt [Sat, 23 Jan 2016 11:30:30 +0000 (12:30 +0100)]
Add new "mirror" cronscript

This just prepares the mirror tree and pushes the changes to the
mirrors.  It is intended to be used for point releases or in other
situations where a manual mirror push is needed.

8 years agoAdd support for detached upstream signatures
Guillem Jover [Sun, 17 Jan 2016 01:08:12 +0000 (02:08 +0100)]
Add support for detached upstream signatures

Upstream tarballs usually come with detached signatures, which would be
useful to have in the source package, as an additional check that could
be performed to verify its integrity and provenance.

dpkg-source supports this since 1.17.20.

Closes: #759401
8 years agoFix proposed-updates handling in byhand-di and byhand-win32-loader
Julien Cristau [Tue, 19 Jan 2016 22:43:10 +0000 (23:43 +0100)]
Fix proposed-updates handling in byhand-di and byhand-win32-loader

To: debian-dak@lists.debian.org
Cc: kibi@debian.org, Julien Cristau <jcristau@debian.org>
Date: Tue, 19 Jan 2016 11:40:01 +0100 (12 hours, 2 minutes, 47 seconds ago)
Resent-From: debian-dak@lists.debian.org

stable-proposed-updates is called "proposed-updates" in projectb, so
that's what we pass to the byhand script.  It shouldn't be turned into
"proposed-updates-proposed-updates".

Signed-off-by: Julien Cristau <jcristau@debian.org>
8 years agoReject arch:all binNMUs
Ansgar Burchardt [Mon, 18 Jan 2016 19:28:31 +0000 (20:28 +0100)]
Reject arch:all binNMUs

While arch:all binNMUs might work for some packages, they still tend
to create problems due to versioned dependencies using

  arch-all-package (= ${source:Version})

as these will be broken by the binNMU.  To avoid these, we reject
arch:all binNMUs for now.

8 years agoRun new SuiteCheck as a per-suite check
Ansgar Burchardt [Sat, 16 Jan 2016 16:58:36 +0000 (17:58 +0100)]
Run new SuiteCheck as a per-suite check

8 years agoDebug suites might also miss the source package
Ansgar Burchardt [Sat, 16 Jan 2016 15:14:06 +0000 (16:14 +0100)]
Debug suites might also miss the source package

If a binary-only upload including debug packages gets accepted from a
policy queue, the source might be missing from the debug suite.

Reported-by: Matthias Klumpp <mak@debian.org>
8 years agoDo not consider MD5-based signatures valid
Ansgar Burchardt [Sat, 16 Jan 2016 14:48:11 +0000 (15:48 +0100)]
Do not consider MD5-based signatures valid

8 years agoDo not accept direct uploads to new build queues
Ansgar Burchardt [Sat, 16 Jan 2016 14:28:44 +0000 (15:28 +0100)]
Do not accept direct uploads to new build queues

8 years agoAdd option to forbid source and/or binary uploads to a suite
Ansgar Burchardt [Sat, 16 Jan 2016 14:23:39 +0000 (15:23 +0100)]
Add option to forbid source and/or binary uploads to a suite

This is intended to be used to forbid uploads to policy queues and build
queues which would result in confusing results.  While we already have a
"reject" option in the dak.conf's SuiteMappings, having the option in
the database means we don't have to edit dak.conf when configuring new
suites.

Another usecase will be the "unstable" suite for Debian Ports: here we
want to import the source packages (and arch:all) from the main archive
and only accept binary uploads.  Additional source packages must go into
the "unreleased-*" suites.  This was not possible via SuiteMappings.

Bug: https://bugs.debian.org/783115

8 years agoCheck for debug suite for upload suite, not override suite
Ansgar Burchardt [Sat, 16 Jan 2016 14:07:47 +0000 (15:07 +0100)]
Check for debug suite for upload suite, not override suite

Debug packages are exempt from NEW if the target suite has a seperate
debug suite.  However the code checked if the target's override suite
had a debug suite...  For the Debian archive this went unnoticed as
experimental's override suite is unstable and both suites have a
seperate debug suite.

Reported-by: Matthias Klumpp <mak@debian.org>
8 years agoLink tracefiles into webview
Joerg Jaspert [Tue, 5 Jan 2016 22:12:08 +0000 (23:12 +0100)]
Link tracefiles into webview

8 years agoAlso create Date: line in archives tracefiles
Joerg Jaspert [Mon, 4 Jan 2016 21:47:21 +0000 (22:47 +0100)]
Also create Date: line in archives tracefiles

8 years agoCreate archive serial
Joerg Jaspert [Mon, 4 Jan 2016 21:44:44 +0000 (22:44 +0100)]
Create archive serial

8 years agoLink "master" to the tracefile, to have one name only
Joerg Jaspert [Mon, 4 Jan 2016 21:17:13 +0000 (22:17 +0100)]
Link "master" to the tracefile, to have one name only

8 years agoMerge remote-tracking branch 'dktrkranz/fixes'
Ansgar Burchardt [Mon, 4 Jan 2016 10:19:28 +0000 (11:19 +0100)]
Merge remote-tracking branch 'dktrkranz/fixes'

8 years agoDefine correct RM version when binary name is different than source
Luca Falavigna [Sun, 3 Jan 2016 18:41:35 +0000 (18:41 +0000)]
Define correct RM version when binary name is different than source

8 years agoStyle foo
Joerg Jaspert [Sat, 2 Jan 2016 22:28:19 +0000 (23:28 +0100)]
Style foo

8 years agoNew function for tempfile generation
Joerg Jaspert [Sat, 2 Jan 2016 22:25:02 +0000 (23:25 +0100)]
New function for tempfile generation

Especially proper adding to the TMPFILES variable to get rid of em
again.

8 years agoRemove dead code
Joerg Jaspert [Sat, 2 Jan 2016 21:46:04 +0000 (22:46 +0100)]
Remove dead code

8 years agoMake timestamping configurable and only let dinstall have it for now
Joerg Jaspert [Sat, 2 Jan 2016 13:38:26 +0000 (14:38 +0100)]
Make timestamping configurable and only let dinstall have it for now

8 years agoMerge remote-tracking branch 'DktrKranz/fixes'
Joerg Jaspert [Sat, 2 Jan 2016 13:34:47 +0000 (14:34 +0100)]
Merge remote-tracking branch 'DktrKranz/fixes'

* DktrKranz/fixes:
  Removal dep check only considers latest sources (Closes: #792578)
  Allow closing bugs if more than one source version is available.

Signed-off-by: Joerg Jaspert <joerg@debian.org>
8 years agoRemoval dep check only considers latest sources (Closes: #792578)
Joachim Breitner [Sat, 2 Jan 2016 13:21:27 +0000 (13:21 +0000)]
Removal dep check only considers latest sources (Closes: #792578)

8 years agoProper timestamps in the logs
Joerg Jaspert [Sat, 2 Jan 2016 13:20:31 +0000 (14:20 +0100)]
Proper timestamps in the logs

8 years agoEntirely make cronscript magic
Joerg Jaspert [Sat, 2 Jan 2016 13:04:55 +0000 (14:04 +0100)]
Entirely make cronscript magic

There is no longer a defined list of arguments it can handle.
Instead the arguments are defined in the var file, whose location
defaults to the FTPMaster location - but can be overriden using an
environment variable.

To be able to have specific types of cronscripts run pre/post actions,
we now check for the existance of preconscript and postcronscript
functions, and if they exist, they are run as the first/last thing
before/after anything listed in the .task file

8 years agoAllow closing bugs if more than one source version is available.
Luca Falavigna [Sat, 2 Jan 2016 12:50:01 +0000 (12:50 +0000)]
Allow closing bugs if more than one source version is available.

8 years agoOnly hardcode path to config if variable isn't set
Joerg Jaspert [Fri, 1 Jan 2016 21:39:55 +0000 (22:39 +0100)]
Only hardcode path to config if variable isn't set

That way we can just set the variable in the crontab file (or users
environment), and do not need to change cronscript (eg. when run on the
security archive)

8 years agoUpdate crontab file to match reality
Joerg Jaspert [Fri, 1 Jan 2016 21:39:41 +0000 (22:39 +0100)]
Update crontab file to match reality

8 years agoMove the reboot-lock handling into the cronscript and out of crontab
Joerg Jaspert [Fri, 1 Jan 2016 13:11:41 +0000 (14:11 +0100)]
Move the reboot-lock handling into the cronscript and out of crontab

8 years agoThose who take a lock shall give it back later
Joerg Jaspert [Fri, 1 Jan 2016 09:27:29 +0000 (10:27 +0100)]
Those who take a lock shall give it back later

8 years agoThere can be only one
Joerg Jaspert [Fri, 1 Jan 2016 09:22:32 +0000 (10:22 +0100)]
There can be only one

Ensure that only one copy of a given crontype can run.

8 years agos/true/false/
Joerg Jaspert [Thu, 31 Dec 2015 22:10:25 +0000 (23:10 +0100)]
s/true/false/

8 years agoMove 3 update- scripts into dinstall.functions/tasks proper
Joerg Jaspert [Thu, 31 Dec 2015 22:05:40 +0000 (23:05 +0100)]
Move 3 update- scripts into dinstall.functions/tasks proper

8 years agoAdjust ERR state for many functions
Joerg Jaspert [Thu, 31 Dec 2015 21:58:55 +0000 (22:58 +0100)]
Adjust ERR state for many functions

Somehow they got turned around and false, but for those we want
it true - and actually error out when they have trouble

8 years agoOnly have an underscore in stagefile if args is non-empty
Joerg Jaspert [Thu, 31 Dec 2015 21:23:56 +0000 (22:23 +0100)]
Only have an underscore in stagefile if args is non-empty

8 years agoRotate REPORT files
Joerg Jaspert [Thu, 31 Dec 2015 13:08:43 +0000 (14:08 +0100)]
Rotate REPORT files

8 years agoAlso allow '-' in suffix of .buildinfo files
Ansgar Burchardt [Thu, 31 Dec 2015 11:31:56 +0000 (12:31 +0100)]
Also allow '-' in suffix of .buildinfo files

This matches what we allow to be used for .changes files in
re_file_changes.

8 years agodaklib/upload.py: Silently accept and discard .buildinfo files
Niels Thykier [Mon, 14 Dec 2015 20:46:28 +0000 (20:46 +0000)]
daklib/upload.py: Silently accept and discard .buildinfo files

Besides some very trivial validation of the filename, accept
.buildinfo files without really looking at them.

Signed-off-by: Niels Thykier <niels@thykier.net>
8 years agoMerge remote-tracking branch 'nthykier/optimise-dominate'
Ansgar Burchardt [Thu, 31 Dec 2015 11:26:26 +0000 (12:26 +0100)]
Merge remote-tracking branch 'nthykier/optimise-dominate'

8 years agodominate: Do not attempt delete rows during dry-run
Niels Thykier [Thu, 31 Dec 2015 11:14:34 +0000 (11:14 +0000)]
dominate: Do not attempt delete rows during dry-run

Without the proper permissions, this will fail even if the transaction
would always be rolled back later.

8 years agodominate: Use a bulk DELETE
Niels Thykier [Thu, 31 Dec 2015 11:12:33 +0000 (11:12 +0000)]
dominate: Use a bulk DELETE

Signed-off-by: Niels Thykier <niels@thykier.net>
8 years agodominate: Do not open log files with -n
Niels Thykier [Tue, 29 Dec 2015 22:02:40 +0000 (22:02 +0000)]
dominate: Do not open log files with -n

Signed-off-by: Niels Thykier <niels@thykier.net>
8 years agoArchive queue/done in monthly cronjob
Joerg Jaspert [Wed, 30 Dec 2015 22:55:36 +0000 (23:55 +0100)]
Archive queue/done in monthly cronjob

archive the directory from 2 months ago, leaving only the last
month accessible directly

8 years agoAdd yearly cronjob
Joerg Jaspert [Wed, 30 Dec 2015 22:25:34 +0000 (23:25 +0100)]
Add yearly cronjob

8 years agoAdjust common functions
Joerg Jaspert [Wed, 30 Dec 2015 22:15:51 +0000 (23:15 +0100)]
Adjust common functions

Mostly logging
Also dont call process-{upload,commands} if there is no file waiting for them.

8 years agoHandle yearly cronjobs
Joerg Jaspert [Wed, 30 Dec 2015 14:14:42 +0000 (15:14 +0100)]
Handle yearly cronjobs

8 years agoA bit more logging, and newoverview in hourly runs backgrounded
Joerg Jaspert [Wed, 30 Dec 2015 14:04:44 +0000 (15:04 +0100)]
A bit more logging, and newoverview in hourly runs backgrounded

8 years agoMerge remote-tracking branch 'lamby/link-homepage' into dmerge
Joerg Jaspert [Wed, 30 Dec 2015 13:47:06 +0000 (14:47 +0100)]
Merge remote-tracking branch 'lamby/link-homepage' into dmerge

* lamby/link-homepage:
  Make homepage URLs (nofollow) links.

Signed-off-by: Joerg Jaspert <joerg@debian.org>
8 years agoVarious
Joerg Jaspert [Wed, 30 Dec 2015 13:17:00 +0000 (14:17 +0100)]
Various

- reorder global variables
- use UTF-8 locale by default
- declare osme variables readonly
- a bit more comments
- remove dangling logfile symlinks at end of cronscript
- indentation fixups

8 years agoAnd away with the old scripts
Joerg Jaspert [Tue, 29 Dec 2015 23:50:16 +0000 (00:50 +0100)]
And away with the old scripts

8 years agoLoadsa changes to make new cronscript way work
Joerg Jaspert [Tue, 29 Dec 2015 23:42:34 +0000 (00:42 +0100)]
Loadsa changes to make new cronscript way work

- Move functions so they are seen,
- Emit logs not only when called by dinstall
- Use correct variables (TMPFILES, not TEMPFILES)
- Define used variables
- create a scriptname link to the logfiles with datebased names
   - in the logdir, not in the configdir
- move linkmorgue from daily to weekly
- remove nonexistant tasks
- make purgeempty and fixsymlinks work on all archives, not just main
- all weekly tasks run in parallel

8 years agoRemember if a file is not on snapshot, don't try every run
Joerg Jaspert [Tue, 29 Dec 2015 14:32:56 +0000 (15:32 +0100)]
Remember if a file is not on snapshot, don't try every run

8 years agoAdjust crontab
Joerg Jaspert [Tue, 29 Dec 2015 13:19:09 +0000 (14:19 +0100)]
Adjust crontab

8 years agoMake homepage URLs (nofollow) links.
Chris Lamb [Tue, 29 Dec 2015 12:26:41 +0000 (13:26 +0100)]
Make homepage URLs (nofollow) links.

Signed-off-by: Chris Lamb <lamby@debian.org>
8 years agoAdjust cronscripts to end up with just one
Joerg Jaspert [Mon, 28 Dec 2015 14:30:44 +0000 (15:30 +0100)]
Adjust cronscripts to end up with just one

8 years agoSort extrafiles
Joerg Jaspert [Mon, 28 Dec 2015 12:45:19 +0000 (13:45 +0100)]
Sort extrafiles

8 years agoGenerate "extrafiles", (closes: #752134)
Joerg Jaspert [Sun, 27 Dec 2015 14:03:51 +0000 (15:03 +0100)]
Generate "extrafiles", (closes: #752134)

A clearsigned sha256sum of files not otherwise having a signed checksum
available.