logging

Imagine the scenario where you have an endpoint or server running and you don’t have centralised logging or visualisation of log data and you need to perform some rapid analysis without wanting to stand up a new set of VMs or services, well this is where cloud really can come into it’s own.

Very rapidly we can setup a Datadog account. (this blog will be updated as I deploy and configure)

Graphical user interface, text, application, email

Description automatically generated

We can deploy the agent in Ubuntu (for example) in a one liner:

Text

Description automatically generated

And then we are ready! Note you can also build from source!

Graphical user interface, text, application, email

Description automatically generated

Ok so we are sending some data, but we haven’t configured sources etc.

Graphical user interface, application

Description automatically generated

If we navigate to logs we can look to setup some sources on our agent.

Let’s install apache2 quickly on the VM.

Install the datadog agent

So the agent config is held here:

sudo nano /etc/datadog-agent/datadog.yaml

Logs are disabled by default

Let’s find the line number

Graphical user interface, text

Description automatically generated

sudo nano +737 /etc/datadog-agent/datadog.yaml

Let’s remove the #

Text

Description automatically generated

Text

Description automatically generated

And change false to true

And save the file

Now lets enable apache logs in the apache.d/conf.yaml file

Copy the example file

sudo cp apache.d/conf.yaml.example apache.d/conf.yaml

Remove the hashes:

Graphical user interface, text

Description automatically generated

Text

Description automatically generated

Restart the agent

service datadog-agent restart

Graphical user interface, text, application, chat or text message

Description automatically generated

Let’s check the status

This says the log file does not exist:

Text

Description automatically generated

Well, let’s check that:

Text

Description automatically generated

Let’s give the agent permissions

Text

Description automatically generated

sudo chmod 655 -R /var/log/apache2/

sudo setfacl -m u:dd-agent:rx /var/log/apache2/access.log

sudo setfacl -m u:dd-agent:rx /var/log/apache2/error.log

sudo setfacl -m u:dd-agent:rx /var/log/apache2/*

Check the agent status:

Text

Description automatically generated

Graphical user interface, text, application

Description automatically generated

We can now explore the logs:

A picture containing chart

Description automatically generated

Now one thing we should think about immediately is post logging! But for now it’s time for a tea!

Ok a few tea’s complete. We’ve now deployed a second VM and attached this. Again we installed apache.

A key thing we need to look at is WHAT logs are we collecting? WHAT scenarios do they cater for? (e.g. this doesn’t capture post requests at the minute). We’ve also go to think about costs, we’ve not only added logs we’ve also enabled the platform security feature. We are on a 14 day trial but 14 day’s isn’t going to get us through the excercise. The VM costs are about £8 per month each. We can spin some up in other clouds as well. Say we allocate £10 for IaaS costs per month per VM. We need to then configure our logs (and tune) we also need to then baseline costs. Let’s see how this goes!

Leave a Reply

Your email address will not be published. Required fields are marked *