Graylog Hitchhiker's Guide To The Galaxy

User Manual: Pdf

Open the PDF directly: View PDF .
Page Count: 17

Download
Open PDF In Browser	View PDF

Graylog Hitchhiker's
Guide to the Galaxy
Resources/Sources

3

Logging basics

3

What to search

3

Graylog components

3

Resource/sources

3

Graylog stream

4

Graylog inputs

4

Graylog alerts

5

Graylog Extractors/grok patterns

6

Install/Setup Graylog
Install/Setup Graylog

7
7

Install/Setup MongoDB

7

Install/Setup Elasticsearch

8

Install/Setup Graylog

8

Install/Setup Nginx and OpenSSL

9

Install/Setup FirewallD

9

Graylog Client Setup

10

Install/Setup Filebeat on CentOS 7 64-bit

10

Install/Setup Packetbeat on Centos 7 64-bit

11

Install/Setup Winlogbeat on Windows 7 64-bit

11

Graylog searching How-to

11

Resources/sources

11

Search by timeframe

12

String based searches

12

Search by key:value pair

14

Multiple key:value pairs

14

Graylog Create an alert
Resources/sources

15
15

Create Graylog stream to filter logs

15

Creating alert for unauthorized SSH login

16

Resources/Sources
●
●
●
●

http://docs.graylog.org/en/2.2/
http://edbaker.weebly.com/blog/windows-and-logstash-quick-n-dirty
https://github.com/elastic/beats/blob/master/winlogbeat/docs/getting-started.asciidoc
http://docs.graylog.org/en/2.2/pages/getting_started/stream_alerts.html

Logging basics
●
●

Log - A saved event of an observable occurrence in an information system that actually
happened at some point in time.
Elements of a log
○ Who, When, or What performed the activity
○ Type of action
■ Ex: Authorize, create, read, update, delete, and accept such as a network
connection
○ Identifiers - Files accessed, query parameters, and etc
○ Before and after values of action performed

●

What to search
●

●

Step one: know what to look for.
○ It seems simple in theory but in practice/real life it’s not. So break things down.
○ If we know they comprised a Windows system then only look at Windows logs.
○ We may know the account compromised so we can lookup authorized user
logins.
○ We may know the data they exfiltrated from the network then we can look up who
has access to that file.
When logging is done correctly we should have records of all events and the before and
after value/result of each event.

Graylog components
Resource/sources
●

http://docs.graylog.org/en/2.2/pages/streams.html

Graylog stream
●
●

Graylog > Streams

Graylog streams​ are a mechanism to route messages into categories in realtime
while they are processed. You define rules that instruct Graylog which message
to route into which streams. Imagine sending these three messages to Graylog

Graylog inputs
●
●
●

Graylog > System > inputs
This will tell Graylog to accept the log ​messages​.
Input types
○ Beats (Filebeat, Packetbeat, Winlogbeat)
○ Syslog
○ Json via HTTP
○ GELF

Graylog alerts
●
●
●

Graylog > Alerts tab
Instruct Graylog to create ​alerts​ or send e-mail notifications when predefined events
meet condition(s).
Alert conditions are based off Graylog streams

Graylog Extractors/grok patterns
●
●

Graylog > System> Grok patterns

Extractors​ allow you to instruct Graylog nodes about how to extract data from
any text in the received message (no matter from which format or if an already
extracted field) to message fields.
○

There are a lot of analysis possibilities with full text searches but the real power
of log analytics unveils when you can run queries like
“http_response_code:>=500 AND user_id:9001” to get all internal server errors
that were triggered by a specific user.

● Grok debugger/creator

Install/Setup Graylog
Install/Setup Graylog
1.
2.
3.
4.

yum update -y && yum install upgrade -y
yum install epel-release -y && yum update -y
yum install -y vim net-tools pwgen
yum install java-1.8.0-openjdk-headless.x86_64

Install/Setup MongoDB
5. cat > /etc/yum.repos.d/mongodb-org-3.2.repo << EOF

[mongodb-org-3.2]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/\$releasever/mongodb-org/3.2/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-3.2.asc
EOF
6. yum install -y mongodb-org
7. chkconfig --add mongod
8. systemctl daemon-reload
9. systemctl enable mongod.service
10. systemctl start mongod.service

Install/Setup Elasticsearch
1. rpm --import ​https://packages.elastic.co/GPG-KEY-elasticsearch
2. cat > /etc/yum.repos.d/elasticsearch.repo << EOF
[elasticsearch-2.x]
name=Elasticsearch repository for 2.x packages
baseurl=https://packages.elastic.co/elasticsearch/2.x/centos
gpgcheck=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1
EOF
3. yum install -y elasticsearch
4. yum update --exclude=elasticsearch-2.x
a. GRAYLOG NEEDS ELASTICSEARCH 2
5. sed -i ‘s/# cluster.name: my-application/cluster.name: graylog/g’
/etc/elasticsearch/elasticsearch.yml
6. chkconfig --add elasticsearch
7. systemctl daemon-reload
8. systemctl enable elasticsearch.service
9. systemctl restart elasticsearch.service

Install/Setup Graylog
1. rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-2.1-repository_latest.rpm
2. yum install graylog-server
3. echo -n yourpassword | sha256sum
a. Copy output text
4. sed -i ‘s/root_password_sha2 =/root_password_sha2 = /g’
/etc/graylog/server/server.conf

5. pwgen -N 1 -s 96
a. Copy output text
6. sed -i ‘s/password_secret =/password_secret =  /g’
/etc/graylog/server/server.conf
7. chkconfig --add graylog-server
8. systemctl daemon-reload
9. systemctl enable graylog-server.service
10. systemctl start graylog-server.service

Install/Setup Nginx and OpenSSL
1. yum install nginx -y
2. mkdir /etc/nginx/ssl
3. openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/nginx/ssl/nginx.key
-out /etc/nginx/ssl/nginx.crt
4. openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048
5. sed -i -e '38,87d' /etc/nginx/nginx.conf
6. cat > /etc/nginx/conf.d/graylog.conf << EOF
a. See resource above for examples
7. sudo setsebool -P httpd_can_network_connect 1
8. systemctl enable nginx
9. systemctl start nginx

Install/Setup FirewallD
1.
2.
3.
4.
5.
6.
7.
8.

yum install firewalld -y
systemctl enable firewalld
systemctl start firewalld
firewall-cmd --permanent --add-service=ssh
firewall-cmd --permanent --add-service=http
firewall-cmd --permanent --add-service=https
firewall-cmd --permanent --add-port=5044/tcp
firewall-cmd --reload

Graylog Client Setup
Install/Setup Filebeat on CentOS 7 64-bit
●

Filebeat helps you keep the simple things simple by offering a lightweight way to forward
and centralize logs and files.

●
1.
2.
3.

Operating System: Linux and Windows
yum update -y && yum upgrade -y
sudo rpm --import ​https://packages.elastic.co/GPG-KEY-elasticsearch
cat > /etc/yum.repos.d/elastic.repo << EOF
[elastic-5.x]
name=Elastic repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF

4.
5.
6.
7.

yum install filebeat -y
mkdir /etc/filebeat/conf.d/
cp /etc/filebeat/filebeat.yml /etc/filebeat/filebeat.yml.bak
cat > /etc/filebeat/filebeat.yml << EOF
filebeat:
registry_file: /var/lib/filebeat/registry
config_dir: /etc/filebeat/conf.d
output.logstash:
hosts: [":5044"]
EOF

8. cat > /etc/filebeat/conf.d/logging.yml << EOF
filebeat.prospectors:
- paths:
- /var/log/*
input_type: log
EOF
9. systemctl enable filebeat
10. systemctl start filebeat

Install/Setup Packetbeat on Centos 7 64-bit
●
●

Packetbeat is a lightweight network packet analyzer that sends data to Logstash or
Elasticsearch.
Operating system: Linux and Windows

Install/Setup Winlogbeat on Windows 7 64-bit
●
●
1.
2.
3.
4.

5.
6.

7.

8.

Winlogbeat
Operating system: Windows
Download the Winlogbeat zip file from the ​downloads page​.
Extract the contents into C:\Program Files.
Rename the winlogbeat- directory to Winlogbeat.
Open a PowerShell prompt as an Administrator (right-click on the PowerShell icon and
select Run As Administrator).
a. If you are running Windows XP, you may need to download and install
PowerShell.
cd 'C:\Program Files\Winlogbeat'
.\install-service-winlogbeat.ps1
a. Run ExecutionPolicy UnRestricted
Edit ​winlogbeat.yml
a. Comment out “#output.elasticsearch:
#hosts:
# - localhost:9200”
b. Uncomment “output.elasticsearch:
hosts:
- :9200”
c. Save, exit
Start-Service winlogbeat

Graylog searching How-to
Resources/sources
●

https://www.youtube.com/watch?v=vxmAIDZe1j0

Search by timeframe
1. Select the “Search” tab
2. Select the drop menu for time

String based searches
● Let’s search for domain names
1. Enter “google.com” into the search.
2. Select the search icon

3. This search will populate results within the specified time frame

4. Select an entry from above to see all the detail
a. We can see below that all entries are a key:value pair

Search by key:value pair
● Let’s search for every log entry where the key is query and the value is google.com
1. Enter “query:google.com” into the search
2. Select the search icon

3. The search will populate entries

Multiple key:value pairs
●

Let’s search for every log entry where the key is and the values google.com

Graylog Create an alert
Resources/sources
●

http://docs.graylog.org/en/2.2/pages/getting_started/stream_alerts.html

Create Graylog stream to filter logs
1. Login into graylog
2. Select “Stream” tab
3. Select “Create Stream”
a. Enter “SSH Unauthorized Access Stream” for title
b. Enter “Failed login attempts for ssh” for description
c. Select “Default index” for index

d. Select “Save”
4. Select “Manage rules” for “SSH Unauthorized Access Stream”
5. Select “Add stream rule”
a. Enter “message” for field
b. Select “contain” for type

c. Enter “pam_unix(sshd:auth): authentication failure” for value

d. Select “Save”
6. Select “I’m done”

Creating alert for unauthorized SSH login
1.
2.
3.
4.

Log into graylog
Select the “Alert” tab
Select “Manage conditions”
Select “Add new condition”
a. Select “SSH Unauthorized Access Stream” for Alert of stream
b. For this example select “Message Count Alert Condition”

c. Select “Add alert condition”
5. Field content for rule

a.
b.
c.
d.
e.

Enter “SSH Unauthorized Access Alert” for title
Enter “5” for time range events occur in
Select “more than” for threshold type
Enter “3” for threshold
Enter “15” for grace period

f. Select “Save”
6. Select “Alert” tab



---

Source Exif Data:

File Type                       : PDF
File Type Extension             : pdf
MIME Type                       : application/pdf
PDF Version                     : 1.5
Linearized                      : Yes
Producer                        : Skia/PDF m67
Page Count                      : 17

EXIF Metadata provided by EXIF.tools