Project: refs

Lightweight, plain text analysis of Apache's access_log.

title: refs
platform: perl
created: 2002
tags: perl, web, statistics, console

Download (tgz, 5KB)

An ancient piece of perl, hacked together aeons ago to provide a simple method of examining recent web traffic statistics. It's ugly and relatively inefficient, but I've used it with great frequency for many years.

Features

  • Lists recent referrers, search engine strings and visitor hostnames
  • Analysis of the previous week's visits, segmented into raw and unique hits, referrals from search engines and external sites, and spider visits.
  • Formatted plaintext output
  • Hyperlinked referrers when used in the context of a CGI script

Usage

tail -2000 access_log | refs

Example Output

---------------------------------+
STATISTICS FOR DOMAIN: erase.net
                 TIME: [17/02 15:35:21]

LAST 10 REFERRERS -----------------------------------------------------------+
  [16/02 20:23:31]        google: daniel jones
  [16/02 20:37:58]  http://forum.hardware.fr/hfr/Hardware/Materiels-problemes-
  [16/02 21:12:23]  http://o-db.com/en/dictionaries/russian_belorussian/
  [16/02 21:49:43]           msn: erase
  [16/02 22:25:59]  http://www.processing.org/reference/libraries/index.html
  [16/02 22:54:09]        google: mpvt vinyl
  [17/02 00:04:57]  http://www.netcraft.com/survey/
  [17/02 00:07:55]  http://processing.org/reference/libraries/index.html
  [17/02 00:19:39]        google: p5_sc daniel jones
  [17/02 00:57:29]        google: erase

LAST 10 SEARCHES -------------------------------------------------------------+
  [16/02 02:25:56]        google: erase
  [16/02 11:26:24]        google: erase
  [16/02 14:05:37]        google: processing supercolider
  [16/02 16:17:06]        google: erase net project
  [16/02 17:08:53]         yahoo: erase
  [16/02 20:23:31]        google: daniel jones
  [16/02 21:49:43]           msn: erase
  [16/02 22:54:09]        google: mpvt vinyl
  [17/02 00:19:39]        google: p5_sc daniel jones
  [17/02 00:57:29]        google: erase

LAST 10 REFERRING SITES ------------------------------------------------------+
  [16/02 16:54:05]  http://forum.hardware.fr/hfr/Hardware/Materiels-problemes-
  [16/02 17:18:48]  http://processing.org/reference/libraries/
  [16/02 18:02:13]  http://forum.hardware.fr/hfr/Hardware/Materiels-problemes-
  [16/02 18:30:52]  http://processing.org/reference/libraries/
  [16/02 19:51:21]  http://processing.org/reference/libraries/index.html
  [16/02 20:37:58]  http://forum.hardware.fr/hfr/Hardware/Materiels-problemes-
  [16/02 21:12:23]  http://o-db.com/en/dictionaries/russian_belorussian/
  [16/02 22:25:59]  http://www.processing.org/reference/libraries/index.html
  [17/02 00:04:57]  http://www.netcraft.com/survey/
  [17/02 00:07:55]  http://processing.org/reference/libraries/index.html

LAST 10 VISITORS ------------------------------------------------------------+
  [17/02 01:28:32]  74.6.28.224
  [17/02 02:09:06]  122.152.128.15
  [17/02 02:28:06]  74.6.7.126
  [17/02 02:44:11]  74.6.29.37
  [17/02 02:53:53]  65.55.104.12
  [17/02 02:54:26]  65.55.208.76
  [17/02 03:29:33]  66.249.65.52
  [17/02 03:53:35]  74.6.25.224
  [17/02 03:53:35]  74.6.19.103
  [17/02 04:03:55]  65.55.104.156

LAST 5 SPIDERS --------------------------------------------------------------+
  [13/02 04:35:23]  voilabot
  [15/02 04:39:12]  gigablast
  [16/02 20:39:02]  askjeeves
  [17/02 03:29:33]  google
  [17/02 03:53:35]  inktomi

PREVIOUS 7 DAYS
+-------+---------------+------------------------+----------------------------+
|       | HITS:         | REFERRERS:             | MISC:                      |
| DATE  | RAW    | UNIQ | SEARCH | SITE  | NONE  | SPIDER/GB | FAVS | DATA    |
+-------+--------+------+--------+-------+-------+-----------+------+---------+
| 02/17 |    142 |   12 |      2 |     2 |     8 |   31/28   |    3 |   1.6Mb |
| 02/16 |    697 |   68 |      8 |    18 |    42 |   23/13   |   23 |   5.8Mb |
| 02/15 |    944 |   67 |      9 |    16 |    42 |  153/144  |   27 |   7.7Mb |
| 02/14 |    939 |   66 |      5 |    17 |    44 |   39/29   |   24 |  12.3Mb |
| 02/13 |   1193 |   90 |      5 |    29 |    56 |   21/15   |   31 |  37.5Mb |
| 02/12 |    964 |   73 |     11 |    22 |    40 |   47/43   |   15 |  35.9Mb |
| 02/11 |   1079 |   89 |      9 |    45 |    35 |   20/12   |   27 |  10.5Mb |
+-------+--------+------+--------+-------+-------+-----------+------+---------+