NEWSYSLOG ToDo -- Things that could be fixed/improved/added. Patches to close any of these items will be gratefully accepted! Other suggestions and comments are also encouraged. - think about loading alternate config files (FreeBSD has this now?) - what about changing ownerships and permissions of archived logs? (i.e. if the permissions are changed in the config file after some archived log files exist) - what about extra copies of archived logs that'll never be overwritten, eg. if the count is reduced? Use glob(3) to find all matching ones? - consider adding a wildcard filename that can stand in for any files in /etc/syslog.conf that are not already mentioned in newsyslog.conf so that the simplest newsyslog.conf could be just the one line: _SYSLOG_ root:wheel 640 10 * D/0 HUP - also think (but not too hard, this is ugly) about supporting the FreeBSD 'G' (glob(3)) flag. Yuck. This could get wildly out of control! But it would make it easier to manage the logs of some of those poorly designed RADIUS daemons.... - consider using date&time stamps for archive names. This helps sites which do normal incremental backups that include their log directories since by using date&time stamps instead of sequence numbers the old files remain static and don't have to be backed up multiple times (though the live data may of course). Presumably we use the time of roll-over to create the new name, thus avoiding the issue of archive files that are still being written to after they are rolled over. Do we use UTC/GMT, or system local time, or allow a command-line option for the user to choose? (syslogd normally uses system local time, assuming it has access to /etc/localtime, should newsyslog just do the same?) How do we count the number of archive files and remove the oldest extra one(s)? Do we use glob(3) to match the name pattern we generate? Then we need to sort them by age, using the filename (will glob(3) effectively always do that for us?). Would this effectively solve the problem of removing stale archives if the count has been reduced? With CE_COMPACT do we still compress just the most "recent" archive file (i.e. the equivalent of the new .0 or .1 depending on CE_PLAIN0). Does the CE_PLAIN0 flag no longer rename the most recent file to have an actual .0 suffix, but rather just skip compressing the most recent log, leaving it named with its roll-over time? - fix handling of log file timestamps to take into account timezones. Note that the common log format actually includes " {+-}TZOFF" after the time, but most strptime() implementations don't seem to have a way to parse and use this value. Syslog timestamps are in local time, so watch out for stupid daylight savings confusion too! - think about what to do with the race between newsyslog and syslogd when rolling multiple files: syslogd can still be processing a previous SIGHUP while newsyslog has gone on to rename the next file leaving it non-existant when syslogd goes to try to open it thus causing spurious errors in the syslog logs about missing files. Probably newsyslog should group logfile processing by PID file and only signal each server once after all its files are rolled. - what happens, or should happen, if/when there are multiple PIDs in a pidfile? - consider supporting the specification of multiple pidfiles for a given log for the case where multiple long-running daemons write to the same logfile, eg. as: /var/run/named-outside.pid,/var/run/named-inside.pid Or teach people to either use a proper syslog daemon instead of allowing/configuring their daemons to write directly to any log files, or else teach them to always use separate log files for each instance! - Add one more optional field: the format of the turnover message, with the default to be the "syslog" style: Mar 15 00:00:01 most newsyslog[17295]: logfile turned over Nothing fancy though. Just strftime() formats, plus one magic specifier to represent where the hostname goes (%@), and another magic specifier for the "newsyslog[17295]: logfile turned over" part (%#). (i.e. run the format specifier through a simple replacement editor for the two magic strings, and then pass it to strftime()). The default would thus be something like this: "%b %d %T %@ %#" - Reverse the order of operations when checking the age of the log file such that the timestamp is always read from the first record of non-binary log files using the turnover format as a guide. Only binary files will revert to checking the age of the .0 file. This will make newsyslog less sensitive to the affects of anyone messing with the archive copies, and more accurate for the first go-around on non-syslog format log files. Perhaps this should only be done for files archived at 24hr multiples (i.e. '-m'), and in these cases the turnover message should also be entered immediately in the new log file too. This will ensure that age determination does not rely on when the daemon writes its first log entry. It may make parsing the first entry easier too. One possible limitation here is that the timestamp may have to appear at the beginning of the line (with only constant characters before and within it). The only way around this might be to iterate strptime() starting at each successive character in the string to try and find a parsable date format that matches the turnover format as it would be with the two magic strings stripped off. For example the standard syslog format "%b %d %T %@ %#" would strip down to "%b %d %T ", which would hopefully match at offset 0 into the first log entry. An HTTP Common Log Format turnover message output format might be: "%@ - - [%Y/%m/%d:%T] \"%#\"" This would have to be stripped down to the following for strptime(): "%Y/%m/%d:%T] \"" - add support for specifying size and interval units other than KB & hrs. - consider adding a "-/" (or "-a"?) command that forces CE_SUBDIR for all log files. - With the "/" flag should we keep the ".0" file in the original directory initially if there's a "0" flag too? This would perhaps help mimic its current purpose of not doing anything to the file until after it's been cleanly closed by all processes logging to it (not that moving a file into a directory instead of just renaming it in the same directory should ever cause any problems on any normal POSIX system). - consider deprecating internal compression once sub-directory archiving is implemented since now a separate processing script can safely go about compressing everything in the directory. - implement FreeBSD's CE_SIGNALGROUP (send signal to negated PID) (but what purpose could this possibly have?). - add a '--with-bzip2' configure option (FreeBSD uses 'J' as the flag to indicate bzip2 should be used to compress the log -- but that's kinda silly -- there's really no point to using different compression on different logs) - add a '--with-compress_options=options' option too --with-compress_options=options pass these additional options to compress (or gzip or bzip2) - think about FreeBSD's 'W' (wait) flag to wait for compression jobs for a given entry (but what purpose could this possibly have?). - get rid of the mtab[] array in parse_dwm() and do something more intelligent that accounts for leap years. Perhaps if today is not day_of_month==1 then strftime("%d") for "now"+24h to see if tomorrow is == 1. (need to adjust for interval near midnight because tomorrow might be into the next interval and we'd miss it!) (alternately just always warn about Mxx > 28 and be done with it!) #ident "@(#)newsyslog:$Name: $:$Id: ToDo,v 1.17 2009/07/16 00:32:03 woods Exp $"