Install and Use GoAccess Web Log Analyzer on Ubuntu 20.04

GoAccess is a real-time web server log monitoring tool. It is written in C and uses the popular ncurses library for its console interface, accessible directly from the command line.

It's great because you can SSH into any web server you control and quickly and securely view or analyze relevant statistics. In addition to the command-line console interface, it is also capable of displaying statistics in other formats such as HTML, JSON, and CSV that you can use in other contexts or share with others.

GoAccess can also be a great alternative to client-side analytics tools depending on your needs. It analyzes your server logs directly, so you don't need to download any additional scripts and your data is completely under your control.

In this tutorial, you will install and configure GoAccess for Apache on Ubuntu 20.04 web server. You will access the Apache log files using GoAccess before looking at the available modules and CLI navigation shortcuts.

Step 1 — Installing GoAccess

In this step, you will install the GoAccess tool and its dependencies.

Start by making sure the database and package system are up to date:

sudo apt update
sudo apt full-upgrade

Now it's time to install GoAccess. A version of this tool is available in the Ubuntu repositories, but it's usually not the latest stable release. For example, the latest version of GoAccess at the time of writing is 1.4, while the version available from the Ubuntu 20.04 repositories is 1.3.

To ensure that the latest stable version of GoAccess is installed on your server, you can either compile from source or use the official GoAccess repository on Ubuntu.

Method 1 — Compiling from source

First, install the dependencies required to compile GoAccess from source:

sudo apt install libncursesw5-dev libgeoip-dev libtokyocabinet-dev build-essential

Next, download the latest version of the GoAccess from their official website with the following command:

wget http://tar.goaccess.io/goaccess-1.4.tar.gz
tar -xzvf goaccess-1.4.tar.gz
cd goaccess-1.4/
./configure --enable-utf8 --enable-geoip=legacy

Output
. . .
Your build configuration:

  Prefix         : /usr/local
  Package        : goaccess
  Version        : 1.4
  Compiler flags :  -pthread
  Linker flags   : -lnsl -lncursesw -lGeoIP -lpthread
  UTF-8 support  : yes
  Dynamic buffer : no
  Geolocation    : GeoIP Legacy
  Storage method : In-Memory with On-Disk Persitance Storage
  TLS/SSL        : no
  Bugs           : [email protected]

Run the make command to build the makefile required for installing GoAccess:

make
sudo make install
goaccess --version

Output
GoAccess - 1.4.
For more details visit: http://goaccess.io
Copyright (C) 2009-2020 by Gerardo Orellana

Build configure arguments:
  --enable-utf8
  --enable-geoip=legacy

Method 2 — Using the Official GoAccess Repos

Another way to install GoAccess is to use the official Ubuntu repositories for the program. This method is preferred if you want it to automatically update to a newer version during system upgrades without having to compile from source for each new version. First you need to add the repository to your server:

echo "deb http://deb.goaccess.io/ $(lsb_release -cs) main" | sudo tee -a /etc/apt/sources.list.d/goaccess.list

wget -O - https://deb.goaccess.io/gnugpg.key | sudo apt-key --keyring /etc/apt/trusted.gpg.d/goaccess.gpg add -

sudo apt update

Finally, install GoAccess:

sudo apt install goaccess

GoAccess is now installed on your Ubuntu server. In the next step, you’ll access and edit its configuration file so that you can make changes to how the program runs.

Step 2 — Editing the GoAccess Configuration

GoAccess comes with a configuration file where you can make permanent changes to the operation of the program. You will edit this file to specify the time, date, and log format so that GoAccess knows how to parse the server logs.

The configuration file can be in the form ~/.goaccessrc or %sysconfdir%/goaccess.conf where %sysconfdir% is /etc/, /usr/etc/ or /usr/local/etc/. To know the location of the configuration file on your server, run the following command:

goaccess --dcf

Sample output /etc/goaccess/goaccess.conf

Edit this config file using nano:

sudo nano /etc/goaccess/goaccess.conf

Multiple lines of the file are commented out. To enable an option, remove the first # in front. First enable the time format setting for Apache. This setting specifies the log format time and allows GoAccess to parse all plain text Apache log files that meet the supported formatting criteria.

# The following time format works with any of the
# Apache/NGINX's log formats below.
#
time-format %H:%M:%S

Next, you’ll uncomment the Apache date-format setting that specifies the log-format date:

# The following date format works with any of the
# Apache/NGINX's log formats below.
#
date-format %d/%b/%Y

Finally, uncomment the log-format setting. Several lines change this setting and the exact one to uncomment depends on the way your web server is set up. If you have a non-virtual hosts setup, uncomment the following log-format line:

# NCSA Combined Log Format
log-format %h %^[%d:%t %^] "%r" %s %b "%R" "%u"

Step 3 — Accessing NGINX Log Files with GoAccess

The NGINX server grants access to your website and keeps an access log for all incoming HTTP traffic. These records, or log files, are stored on the system and can be a valuable source of information about your website’s usage and audience.

On Ubuntu, the NGINX log files are stored in the /var/log/nginx/access.log directory by default. To inspect the contents of this directory, run the following command:

sudo ls /var/log/nginx/access.log

Let’s go ahead and run GoAccess against the NGINX access logs to gain insight into what type of traffic is being handled by the web server. Run the following command to analyze your access.log file with GoAccess:

sudo goaccess /var/log/nginx/access.log

goacces untuk report bentuk html dan kirim ke email (bisa di buat automatisasi ini contoh command nya)

sudo goaccess -f /var/log/nginx/access.log --log-format="%h %^[%d:%^] \"%r\" %s %b \"%R\" \"%u\""   --date-format="%d/%b/%Y"  --time-format="%T"  -a > report.html

untuk kirim email ini command nya

sudo mutt -s "report-html" -a /home/agungsurya/report.html -c [email protected]`