Logmonster ChangeLog

5.36 - Sep 2014
    version bump, for consistent version numbers in all packages

3.12 - Apr 2013
    testing changes, to resolve test errors on Windows

3.11 - Apr 2013
    testing & packing changes

3.09 - Apr 2013
    packaging changes

3.08 - Dec 2012
    added missing URL dependency
    updated bin/install_deps.pl

3.07 - Oct 2012
    removed Perl.pm
    updated Utility.pm to v 5.35
    stripped out apache httpd.conf parsing. Instead, detect vhosts from the log files (works with any http server)
    added confdir (path to processor config files)
    statsdir is now a full path to a directory
    the 'drop logs into vhost docroot" feature is gone, pending a clever way to reliably deduce the vhost document root for various web servers.
    removed vhost config option
    added default_vhost option
    moved _progress subs into Logmonster.pm
    removed get_domains_list, check_stats_dir, get_domains_list_from_directories, get_vhosts_from_file, install_default_awstats_conf, turn_domains_into_sort_key
    abstracted new subs: get_log_files, spam_check, open_vhost_handle, report_matches, report_spam_hits, report_bad_hits,
    added lighttpd config entries to instructions

3.05 - Sep 2008

    removed inc/* from distribution
    updated Makefile.PL, adding LICENSE and missing dependencies
    moved logmonster.pl into bin dir
    replaced bin/install_freebsd_deps.sh with install_deps.pl
    added 'make deps' target to Makefile.PL

3.04 - Nov 2007

    included "inc"' in modules
    Utility.pm - only attempt to use sudo as a last resort
    Utility.pm & Utility.t- synced with Mail::Toaster
    moved Regexp::Log to inc dir

3.03 - Feb 17, 2007

    Utility.pm - only attempt to use sudo as a last resort
    bin/install_freebsd_deps.sh - install Regexp::Log port

3.02 - Feb 16, 2007

    added support for detecting dynamic mass virtual hosting
    evaluate log for referrer spam and discard the referral spam.
    optional referral spam reporting
    the hits-per-vhost was only counting the number of hits for the master domain. Domain alias hits weren't showing up in the hits-per-vhost report.
    added bin/install_freeebsd_deps.sh installer
    default log location is now /var/log/http (was /var/log/apache)
    added Regexp::Log::Logmonster - uses a compiled regexp to parse log entries (very useful for evaluate for referrer spam).

  3.01  - Feb 2, 2007
    Added Params::Validate dependency to Makefile.PL

    Fixed bug where time offset was being ignored.

    Checks more locations for awstats.pl (needs to become a config file

    Removed an unnecessary dependency on Mail::Toaster in Utility.pm.

  3.00rc2 - Oct 14, 2006
    *   updated usage examples

    *   added FAQ

    *   updated META.yml to new format

    *   email report formatting

        added OUTPUT_AUTOFLUSH b/c printing to *STDERR flushes immediately
        and mixed print statements did not. Email report is better formatted

    *   added inc/Module/Install

    *   added Build.PL

    *   added examples/daily_email_report.txt

    *   Logmonster.pm

        debug was not being set properly, report formatting tweaks

    *   Utility.pm

            added _progress sub

  3.00rc1 - Oct 02, 2006 (no public release)
   massive code overhaul
    This new version is mostly about code quality and maintainability (not
    new features). The large chunks of code have been modularized into
    smaller subroutines and tests have been written to test the
    functionality of each sub. There are now 102 different tests (was 23) in
    the test suite. Added t/Test-coverage.pl, t/Logmonster.t, t/pod.t,
    t/pod-coverage, t/00.load

    Nearly all the "working" code has been moved into
    lib/Apache/Logmonster.pm. Logmonster.pl is now a "shell" consisting of a
    little bit of code and a lot of documentation.

    All the functions are now Object Oriented. Time will tell if that is a A
    Good Thing[TM] but it makes reading the code and understanding where all
    the calls are going much, much easier. added doc/*

    The documentation has been significantly updated, addressing many of the
    common questions and comments I have received.

    The reporting has been overhauled. You still get the same information
    but by default, if everything is okay it runs entirely silently. A
    single -v will output status messages that make for a nice birds eye
    view of your web log traffic. You can add additional -v options for even
    more verbose reporting.

    Interface change: instead of -m for month, -d for day, and -h for hour,
    you use a -i [hour|day|month] option. The old -mdh options are
    officially deprecated but will continue to work for the indefinte

    synced lib/Apache/Logmonster/Perl & Utility with Mail::Toaster 5

  2.77 - no public release
    *   create daily userlogs

        new feature submitted by Gernot Hueber.

    *   autogenerate vhost stat dirs

        new feature funded by Lewis Bergman: statsdir can be automatically
        created if it does not exist. Set statsdir_policy = create in
        logmonster.conf to enable this feature.

    *   incremental awstats config files.

        New feature by Lewis: Instead of having a full awstats.conf file for
        each vhost, each vhost has its own file that contains only specific
        info for that vhost and includes the generic
        /etc/awstats/awstats.conf file with the "global" settings.

    *   added loguser and loggroup to .conf

  2.76 - Jun 04, 2005
    statsdir can now be an absolute path
        statsdir can now be an absolute path (ex: /var/www/html). If so,
        processor output will be stored in statsdir/vhost. Otherwise, it
        works as it used to and output goes to vhost/statsdir. This is
        useful if you have a seperate machine (not the web server) that does
        the processing and that system does not have access to the vhost

  2.75 - Feb 05, 2005
    * you can now select which stats processor is used for each virtual
    host. Create a .processor file in the stats dir and place the name of
    the processor on the first line.

    * test to make sure the log file exists before trying to compress it
    (supresses spurious warnings)

    * If vhosts are all in a directory, skip any files that end with ~ (vim)
    or .bak (user).

    * the code that hashed the domain list collected the settings from
    httpd.conf and stuffed them into a hash with the name of the vhost. This
    worked great, as long as ServerName was the first declaration in your
    httpd.conf. I discovered this fails otherwise. Now it stuffs the
    settings into a hash and then, after all the data is collected, moves
    the data into a new hash keyed off the vhost servername. The function
    now works regardless of order in the vhost container.

    * If a perl module was missing, the script would fail after attempting
    to load Mail::Toaster::Perl (which may not exist) Added
    Apache::Logmonster::Perl to distro

  2.74  - Nov 26, 2004
     package is now named Apache::Logmonster to fit nicely into a CPAN category
     bundled up for CPAN & freshmeat release
         updated package NAME
         removed MATT::* dependency, added Compress::Zlib dep
         updated package name
       added more example settings to logmonster.conf
       cleaned up pod docs for prettier web page formatting
       added TODO file
       remove MATT::Bundle reference from FAQ

  2.73  - Nov 12, 2004
      updated lib/Logmonster/Utility to latest
      fixed get_the_date bug in Utility
      added many more tests for Utility
      fixed a bug in my fileparse call (File::Basename)

  2.72  - Nov 11, 2004
      Removed MATT::Utility dependency
      added lib/logmonster/Utility (logmonster::Utility)
      removed Exporter
      updates for use with logmonster::Utility
      replaced StripLastDirFromPath with File::Basename

  2.71  - Oct 22, 2004
      Raymond Dijkxhoorn suggested not sorting the files if 
        there is only one host. Shucks, that's a reasonable enough 
        thing to do, for those of you with only one web server. ;-)
      Logmonster will now dutifully skip sorting logs if only
        one hostname is configured in logmonster.conf

  2.70  - 0ct 22, 2004
      allow for ServerName to have a :80 style suffix
      added verbose (-v) flag.
      fixed up reporting so quiet mode is really quiet
         unless there are errors
      normal output is prettier
      debugging output is much prettier
      apache config file parsing is now much more versatile
         if you have a folder full of files for vhosts, you can
         have multiple vhosts within a file now
      misc internal changes for efficiency
      prototyped all subroutines
      added additional comments here and there
      added test for FileHandle

  2.6.6 - Jun 16, 2004
      Added inline documentation to a few of the subs
      Modified SortVhostLogs so that it uses much less memory
         by writing to the log file as we sort instead of building
         an array and then writing the array contents to the file
         in GetDomainList, I forgot to add domains without aliases
         to %domains 
      GetServerName: made the regexp search more reliable (no
         known problems but the potential existed).

  2.6.5 - Jun 10, 2004
     Switched date parsing regexp from / / to /\s+/ in 
       SortVhostLogs per Earl Ruby (eruby@switchmanagement.com)
       for compatibility with Apache::Registry

  2.6.4 - Apr 14, 2004
      Added regexp notes
      Added additional debugging
      Check for files locally before trying to compress them

  2.6.3 - Mar 28, 2004
      Updated regexp to support numeric vhosts
      Updated logmonster.pl pod documentation
      Updated Makefile.PL, added README, FAQ files
      Added BSD copyright

  2.6.2 - Mar 11, 2004
      Corrected a typo in the logmonster.pl config file.

  2.6.1 - Feb 24, 2004
      Fixed problem where paths with caps weren't detected 
       (search string was lower cased)

  2.6   - Feb 21, 2004
      Updated documentation and web site. More informative.

  2.5.2 - Sep 24, 2003
      Added a strip leading spaces function to GetVhostsFromFile
      Adjusted so FindTheBin will find awstats in its default
        location (/usr/local/www/cgi-bin/awstats.pl)

  2.5   - Sep 15, 2003
      Fixed a problem with HitsPerVHost not getting written
      Fixed a mis-feature where running logmonster -r was 
        clobbering the active log processing dir. Oops.
      Fixed a couple problems related to interaction between
        script and MATT::Utility
      Made quite a few failures more graceful. This later proved
        to be only beneficial from a theoretical standpoint
        as it didn't solve the problem I was tracking down.

  2.4   - Aug 14, 2003
      Updated FetchLogFiles so that you don't need to ssh to localhost
      Updated URLs from matt.simerson.net to tnpi.biz
      Moved configuration from script to logmonster.conf
      Added Changes to the CVS depot
      Added support for AWStats log processor
      Added lots of documentation to logmonster.conf

  2.3   - Jul 10, 03
      Added pod documentation.  
      Cleaned up SysCmd calls and now SysCmd calls use MATT::Utility

  2.2   - Mar 24, 03
      Moved many subs out of script into modules

  2.1.1 - Mar 18, 03
      Report hit counts (-r designed to be used with SNMP and RRDutil)

  2.1   - Mar 17, 03
      Moved code out of FetchLogFiles to GetTheLogDir for reuse
      Writes counters to $logdir/HitsPerVhost.txt
      Writes activity log to $logdir/Logmonster.txt
      Renamed $logdir to $logbase

  2.0.3 - Mar 07, 03
      Moved FAQ and Changelog to web site.
      Created web page for logmonster.
      Added support for http-analyze. It should work but I
        haven't used http-analzye in about 5 years so I 
        might be forgetting something.

  2.0.2 - Mar 06, 03
      Added the httpd.conf parsing stuff. Now works with a vhost
        directory or parses out of your httpd.conf - cool :)

  2.0.1 - Mar 06, 03
      Moved vhost log pre-processer checks out of FeedTheMachine
        into CheckStatsDir. Run it before SortVhostLogs so we 
        skip sorting any logs that we aren't going to store.
      Count up invalid lines in log files and report them instead
        of dying when we encounter them.
      Print out prettier logs.

  2.0   - Mar 04, 03
      An entire re-write
      Logic is much cleaner now and way more efficient
      Pulls most settings out of apache config files
      Reporting is much better :)
      Added -b (process archived logs) feature
      Added -h & -m (hourly & monthly processing)
      Added -n (dry run) so you can preview what it'll do
      Now clean enough that I'll make it publicly available

  1.4   - Feb 22, 03
      Major code cleanups
      Reworked fetch_the_files
      Added system_command
      Added FindTheBin

  1.3   - Nov 25, 02
      Added use strict (forced code cleanup)
      Added getopt::std instead of custom parser
      Moved $quiet to $opt_q, $debug to $opt_v

  1.2.1 - Dec 14, 01
      Made it work with matt.simerson.net

  1.2   - Jun 28, 01
      Cleaned up the code, added debug flag, expanded logic
        so v1.2 can replicate 1.0 & 1.1 behaviour with options

  1.1   - Feb 14, 01
      Added support for multiple domains

  1.0   - Oct 29, 00 - Origin