Onderwerp: Webalizer crashes

Waar kan ik logs vinden voor webalizer-crashes?
Één van mijn klanten zeurt 4x per maand dat er een dag mist op zijn webalizer-statistieken, terwijl de statistieken bij andere domeinen wel goed zijn gegaan die nacht.

Hoe achterhaal ik dit?

Genereer de statistieken voor dat ene domein eens handmatig, in debug mode ( -d ) en zie de uitvoer. Misschien een regelmatig terugkerende logfile-regel die Webalizer niet kan parsen?

Dankje voor je bijdrage. Op deze manier webalizer-statistieken generen is echter niet mogelijk in Plesk. Ik heb het volgens de KnowledgeBase gedaan.

Echter het probleem is, het ging deze keer, ondanks foutmeldingen gewoon helemaal goed.

Code:

[root@server06 domein.nl]# /usr/local/psa/admin/sbin/statistics  --calculate-one --domain-name=domein.nl
Webalizer V2.01-10 (Linux 2.6.18-8.el5) English
Using logfile STDIN (clf)
DNS Lookup (15): 649 addresses in 433.50 seconds, 1/sec
Webalizer V2.01-10 (Linux 2.6.18-8.el5) English
Using logfile STDIN (clf)
DNS Lookup (15): None to process
Webalizer V2.01-10 (Linux 2.6.18-8.el5) English
Using logfile STDIN (ftp)
DNS Lookup (15): None to process
Webalizer V2.01-10 (Linux 2.6.18-8.el5) English
Using logfile STDIN (ftp)
DNS Lookup (15): None to process
Webalizer V2.01-10 (Linux 2.6.18-8.el5) English
Using logfile STDIN (clf)
Using DNS cache file /usr/local/psa/var/lib/webalizer/webalizer.cache
Creating output in /home/httpd/vhosts/domein.nl/statistics/webstat
Hostname for reports is 'domein.nl'
Reading history file... webalizer.hist
Reading previous run data.. webalizer.current
Saving current run data... [02/26/2008 16:42:04]
Generating report for February 2008
Generating summary report
Saving history information...
62889 records in 111.00 seconds, 566/sec
Webalizer V2.01-10 (Linux 2.6.18-8.el5) English
Using logfile STDIN (clf)
Using DNS cache file /usr/local/psa/var/lib/webalizer/webalizer.cache
Creating output in /home/httpd/vhosts/domein.nl/statistics/webstat-ssl
Hostname for reports is 'domein.nl'
Reading history file... webalizer.hist
Reading previous run data.. webalizer.current
Saving current run data... [02/26/2008 16:42:14]
Generating report for February 2008
Generating summary report
Saving history information...
26 records in 2.00 seconds, 13/sec
Webalizer V2.01-10 (Linux 2.6.18-8.el5) English
Using logfile STDIN (ftp)
Using DNS cache file /usr/local/psa/var/lib/webalizer/webalizer.cache
Creating output in /home/httpd/vhosts/domein.nl/statistics/anon_ftpstat
Hostname for reports is 'domein.nl'
History file not found...
Previous run data not found...
No valid records found!
Webalizer V2.01-10 (Linux 2.6.18-8.el5) English
Using logfile STDIN (ftp)
Using DNS cache file /usr/local/psa/var/lib/webalizer/webalizer.cache
Creating output in /home/httpd/vhosts/domein.nl/statistics/ftpstat
Hostname for reports is 'domein.nl'
Reading history file... webalizer.hist
Reading previous run data.. webalizer.current
No valid records found!
statistics: Unable to get dir size of /var/lib/mailman/lists/nieuwsbrief
System error 2: No such file or directory
statistics: Unable to get dir size of /var/lib/mailman/lists/nieuwsbrief
System error 2: No such file or directory
statistics: Unable to get dir size of /var/lib/mailman/archives/private/nieuwsbrief.mbox
System error 2: No such file or directory
statistics: Unable to get dir size of /var/lib/mailman/archives/private/nieuwsbrief.mbox
System error 2: No such file or directory
statistics: Unable to get dir size of /var/lib/mailman/archives/private/nieuwsbrief
System error 2: No such file or directory
statistics: Unable to get dir size of /var/lib/mailman/archives/private/nieuwsbrief
System error 2: No such file or directory
[root@server06 domein.nl]#

Statistieken waren in een halve minuten op basis van 40MB logfiles keurig gegenereerd.

Erg frustrerend. Klant klaagt over statistieken waarbij er soms dagen totaal missen. En ik heb geen idee hoe dat kan.

Het opvallende vind ik dat overal 0 staat gewoon, terwijl de cronjob pas 's nachts rond 2 uur gaat draaien. Hij zou op z'n minst dus al 2 uur van 18 en 21 februari meegenomen moeten hebben.

Backups zijn 's nachts 19 en 22 februari gedraaid (om de 3 dagen), echter heb ik niet het gevoel dat die invloed gehad hebben.

heeft de gebruiker misschien zelf de logbestanden op die dagen verwijderd?
bestanden zijn inderdaad 40meg groot en als gebruikers het mappenoverzicht zien zien ze ook dat die data erg vele ruimte in neemt, misschien heeft de gebruiker er dus gewoon zelf schuld aan.

alleen geroteerde logfiles zijn via plesk panel te verwijderen.

Ik zal het even afwachten. Hopelijk gebeurt het niet snel weer.
Indien wel, dan zal ik SWSoft en Webalizer even e-mailen in de hoop dat zij iets meer informatie kunnen verschaffen. Al verwacht ik zeker van de laatste vrij weinig.

Helaas. Het ging uiteindelijk 2 weken goed en nu twee-maal achter elkaar weer niet.

Ik laat de klant wel awstats gebruiken ipv webalizer. Hopelijk brengt dit beter resultaat.

[root@server06 ~]# /usr/local/psa/admin/sbin/statistics --calculate-one --domain-name=domeinforum.nl | mail dennis@hosting.nl
statistics: Unable to execute query 'SELECT val FROM misc WHERE param='actionlog_rot_type'': MySQL server has gone away
[root@server06 ~]#

Code:

Create/Update database for config "/usr/local/psa/etc/awstats/awstats.domeinforum.nl-http.conf" by AWStats version 6.6 (build 1.887)
From data in log file "-"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 14
 Found 0 dropped records,
 Found 0 corrupted records,
 Found 0 old records,
 Found 14 new qualified records.
Build main page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output
Build alldomains page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=alldomains
Build allhosts page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=allhosts
Build lasthosts page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=lasthosts
Build unknownip page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=unknownip
Build allrobots page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=allrobots
Build lastrobots page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=lastrobots
Build session page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=session
Build urldetail page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=urldetail
Build urlentry page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=urlentry
Build urlexit page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=urlexit
Build osdetail page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=osdetail
Build unknownos page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=unknownos
Build browserdetail page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=browserdetail
Build unknownbrowser page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=unknownbrowser
Build refererse page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=refererse
Build refererpages page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=refererpages
Build keyphrases page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=keyphrases
Build keywords page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=keywords
Build errors404 page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-http -staticlinks -configdir=/usr/local/psa/etc/awstats -output=errors404
20 files built.
Main HTML page is 'awstats.domeinforum.nl-http.html'.
Create/Update database for config "/usr/local/psa/etc/awstats/awstats.domeinforum.nl-https.conf" by AWStats version 6.6 (build 1.887)
From data in log file "-"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 437
 Found 0 dropped records,
 Found 0 corrupted records,
 Found 0 old records,
 Found 437 new qualified records.
Build main page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output
Build alldomains page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=alldomains
Build allhosts page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=allhosts
Build lasthosts page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=lasthosts
Build unknownip page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=unknownip
Build allrobots page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=allrobots
Build lastrobots page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=lastrobots
Build session page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=session
Build urldetail page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=urldetail
Build urlentry page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=urlentry
Build urlexit page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=urlexit
Build osdetail page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=osdetail
Build unknownos page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=unknownos
Build browserdetail page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=browserdetail
Build unknownbrowser page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=unknownbrowser
Build refererse page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=refererse
Build refererpages page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=refererpages
Build keyphrases page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=keyphrases
Build keywords page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=keywords
Build errors404 page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-https -staticlinks -configdir=/usr/local/psa/etc/awstats -output=errors404
20 files built.
Main HTML page is 'awstats.domeinforum.nl-https.html'.
Create/Update database for config "/usr/local/psa/etc/awstats/awstats.domeinforum.nl-anon_ftp.conf" by AWStats version 6.6 (build 1.887)
From data in log file "-"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Jumped lines in file: 0
Parsed lines in file: 0
 Found 0 dropped records,
 Found 0 corrupted records,
 Found 0 old records,
 Found 0 new qualified records.
Build main page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output
Build alldomains page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=alldomains
Build allhosts page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=allhosts
Build lasthosts page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=lasthosts
Build unknownip page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=unknownip
Build allrobots page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=allrobots
Build lastrobots page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=lastrobots
Build session page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=session
Build urldetail page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=urldetail
Build urlentry page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=urlentry
Build urlexit page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=urlexit
Build osdetail page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=osdetail
Build unknownos page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=unknownos
Build browserdetail page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=browserdetail
Build unknownbrowser page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=unknownbrowser
Build refererse page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=refererse
Build refererpages page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=refererpages
Build keyphrases page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=keyphrases
Build keywords page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=keywords
Build errors404 page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-anon_ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=errors404
20 files built.
Main HTML page is 'awstats.domeinforum.nl-anon_ftp.html'.
Create/Update database for config "/usr/local/psa/etc/awstats/awstats.domeinforum.nl-ftp.conf" by AWStats version 6.6 (build 1.887)
From data in log file "-"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 16
 Found 0 dropped records,
 Found 0 corrupted records,
 Found 0 old records,
 Found 16 new qualified records.
Build main page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output
Build alldomains page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=alldomains
Build allhosts page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=allhosts
Build lasthosts page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=lasthosts
Build unknownip page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=unknownip
Build allrobots page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=allrobots
Build lastrobots page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=lastrobots
Build session page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=session
Build urldetail page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=urldetail
Build urlentry page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=urlentry
Build urlexit page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=urlexit
Build osdetail page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=osdetail
Build unknownos page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=unknownos
Build browserdetail page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=browserdetail
Build unknownbrowser page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=unknownbrowser
Build refererse page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=refererse
Build refererpages page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=refererpages
Build keyphrases page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=keyphrases
Build keywords page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=keywords
Build errors404 page: "/var/www/cgi-bin/awstats/awstats.pl" -config=domeinforum.nl-ftp -staticlinks -configdir=/usr/local/psa/etc/awstats -output=errors404
20 files built.
Main HTML page is 'awstats.domeinforum.nl-ftp.html'.
statistics: Unable to execute query 'SELECT val FROM misc WHERE param='actionlog_rot_type'': MySQL server has gone away

Kan het zijn dat de timeout=100 waarde uit /etc/my.cnf hiermee te maken heeft? Ik vind het dan alleen extreem raar dat het bij andere websites met veel grotere logs wel gewoon goed gaat met dezelfde timeout-waarde.

Heb je http://dev.mysql.com/doc/refman/5.0/en/gone-away.html al gelezen?

Wat gebeurt er als deze query handmatig uitvoert?