Follow this guide to get up and running with
Grafana. Once you complete this setup, you'll be collecting and displaying realtime performance metrics from your favorite API.
Note: This is a long read, at ~2300 words. It should take about an hour to get through the deployment. If you need some motivation get coffee'd up and review the example charts at the end of the post.
This post is fully updated to reflect the latest versions. Also, this article was moved from our previous hosting over to ghost. Enjoy!
We cover the topics below. No prior knowledge is required.
- Download CentOS 7 Linux ISO
- Create a virtual machine
- Update CentOS 7
- Configure CentOS 7 (CLI or GUI Desktop)
- Install InfluxDB (time-series database)
- Create two InfluxDB databases (iops and compute)
- Install Grafana (web interface to display our metrics)
- Point Grafana to an InfluxDB datasource
- Suggest techniques to gather datapoints
Note: This can also be deployed on Windows and many other operating systems. See the official InfluxData and Grafana websites for downloads and instructions.
About Collecting Stats
After completing the setup, you can populate datapoints using your own scripts, or something like my vFlux-Stats-Kit on github.
By following this guide exactly, you will have the two databases that the above scripts expect. However, you are free to name the databases as you choose and fill them with any datapoints desired.
These packages run great on the smallest imaginable virtual machine. However, the proper setup should be a minimum of the following:
1 Virtual Machine (minimum)
CentOS 7 (Minimal Install is fine)
2 CPU (4 CPU recommended)
40GB Harddrive* (80GB+ recommended)
Tip: If you need to resize your disk, do so before powering on. For existing virtual machines consult http://kb.vmware.com/kb/1006371
In this exercise, we use a single disk for simplicity. In a high performance deployment, you may want dedicated disks for InfluxDB.
Install CentOS 7
Skip this if you already have the OS up and running.
Step 1. Download
- Download CentOS7 Minimal at https://www.centos.org/download/
- The size is just under 1GB currently.
- We can add packages later.
Step 2. Create
- Create a virtual machine (i.e. on VMware Fusion or VMware vSphere)
- Point the virtual machine to the ISO
- Select the option to edit virtual hardware before powering on
- Remove any extra devices (i.e. Camera, Floppy)
- Size the disk
- Configure CPU count
- Configure Memory
- Optionally, set the BIOS to UEFI
Note: CentOS 7 supports both Legacy BIOS or UEFI
Step 3. Power
- Power On virtual machine
- The CentOS splash screen appears
- Select the option to install to disk
- Choose the guided wizard instead of text only
- Configure network, partitioning, NTP, etc.
- Begin the Install
Note: We are not done yet! While the install runs perform the next step.
Step 4. Users and Passwords
While the install is running, we can perform additional tasks such as setting the root password and creating a user. Click the appropriate icon to perform each of the following:
- Set root password
- Create a user
Tip: Throughout this article, we begin as root for the initial updates, etc. We then start using sudo when adding additional components. Please adjust as needed.
Configure CentOS (CLI or GUI)
We can run in CLI-only mode, or add a desktop environment. You can also start with CLI mode and then add a desktop later. Choose an option below.
Option A - CLI Setup
If you will use CLI only (i.e. no GUI), then follow these steps.
- Login as root
- Configure an IP address using the
nmtuitext-based user interface tool
- Perform a
systemctl restart network
- Test your success so far
- Install VMware Tools with
yum install open-vm-tools
- Update the OS and existing packages with
- [optional] To allow root login via SSH, use the vi editor to uncomment the root login restriction.
We will do this with
- [optional] If you made the change in #7 above, perform a
systemctl restart sshd.service
- If desired, confirm SSH access (i.e. with putty)
Option B - Desktop Setup
To deploy a desktop environment on CentOS 7, follow the steps below.
Here, we start with the CentOS 7 Minimal download. We then update, and add a desktop environment. We expect you to have CentOS 7 installed already, and be logged in as root (or use sudo).
Let's start by updating and adding some packages we need.
yum -y update yum -y install epel-release yum -y install open-vm-tools-desktop
Observe the above; When configuring CentOS 7 for desktop mode, we use
open-vm-toolslike we did for CLI mode earlier.
Next, we will install the KDE desktop environment.
yum -y groups install "KDE Plasma Workspaces"
Note: Replace the portion in quotes with "GNOME Desktop" if you prefer that.
Show Startup (CLI or GUI)
Determine if we are currently configured for CLI or GUI.
Configure Startup Environment
Set the default environment to be GUI.
From a terminal enter the following, or reboot however you like.
Login to Desktop
After the CentOS system reboots, login to the desktop as the user you created during the install. If you did not create a user yet, do that now before proceeding.
From here on, we issue commands with sudo.
Installing InfluxDB on CentOS 7
To install InfluxDB on CentOS, there are a few options. For this example, we will create a yum repository and install from there.
Add InfluxDB to yum
Just paste this entire snippet into your putty or terminal session:
cat <<EOF | sudo tee /etc/yum.repos.d/influxdb.repo [influxdb] name = InfluxDB Repository - RHEL \$releasever baseurl = https://repos.influxdata.com/rhel/\$releasever/\$basearch/stable enabled = 1 gpgcheck = 1 gpgkey = https://repos.influxdata.com/influxdb.key EOF
sudo yum install influxdb
We now have several binaries added to
influx influxd influx_inspect influx_stress influx_tsm
influx is the CLI tool to manage things.
influxd is the actual binary that runs the program itself.
sudo systemctl start influxdb
The influxd Process
Let's observe the influxd process and learn which configuration file it is pointing to:
ps -waux | grep influxd
You should see something like:
/usr/bin/influxd -config /etc/influxdb/influxdb.conf
View the contents:
Or the more official way:
for more detail on using configuration files with InfluxDB, see https://docs.influxdata.com/influxdb/v1.5/administration/config/
Allow InfluxDB through Firewall
sudo firewall-cmd --permanent --zone=public --add-port=8086/tcp sudo firewall-cmd --permanent --zone=public --add-port=8083/tcp sudo firewall-cmd --reload
Configure InfluxDB for Automatic start-up
sudo systemctl enable influxdb.service
To interact with your installation of InfluxDB (i.e. create users, databases, etc.) perform the following:
- Launch terminal from within CentOS or SSH
- Change directory to
influxand hit enter
mike@centos bin]$ influx Connected to http://localhost:8086 version 1.5.2 InfluxDB shell version: 1.5.2 >
Create InfluxDB Users
For this example, create a user called
esx with a password of
esx. Type the following into your influx CLI session and press enter.
CREATE USER esx WITH PASSWORD 'esx' WITH ALL PRIVILEGES
Tip: Influx commands only return interactive messages on failure. So after hitting enter above, if you get no feedback, this is good.
Note: For full details on InfluxDB authentication:
Create InfluxDB Database
For this example, create a database called
iops and another called
CREATE DATABASE iops CREATE DATABASE compute
Show InfluxDB Databases
influx CLI, we can use the
SHOW DATABASES command. We will always see
_internal which is the default used by the system. We can also see the databases we added (
PS /home/mike> influx Connected to http://localhost:8086 version 1.5.2 InfluxDB shell version: 1.5.2 > SHOW DATABASES name: databases name ---- _internal iops compute >
Note: We will use these databases in the next steps.
First we will install dependencies. Since you probably already have these, you will likely see a message saying
nothing to do; this is fine.
sudo yum -y install initscripts fontconfig
Additional font support (recommended)
Add the following packages to allow server-side image rendering.
sudo yum -y install fontconfig freetype* urw-fonts
touch to create a file called
sudo touch /etc/yum.repos.d/grafana.repo
Edit the file with
vi. We will use
sudo vi /etc/yum.repos.d/grafana.repo
i to enter insert mode and then paste in the following contents:
[grafana] name=grafana baseurl=https://packagecloud.io/grafana/stable/el/7/$basearch repo_gpgcheck=1 enabled=1 gpgcheck=1 gpgkey=https://packagecloud.io/gpg.key https://grafanarel.s3.amazonaws.com/RPM-GPG-KEY-grafana sslverify=1 sslcacert=/etc/pki/tls/certs/ca-bundle.crt
To save and exit in
<ESC> on your keyboard and then type:
<Enter> on your keyboard to complete the command.
sudo yum -y install grafana
About Grafana Configuration
If desired, you can modify the Grafana configuration file at
Set Grafana for automatic start-up
sudo systemctl enable grafana-server.service
Start Grafana Now
sudo /bin/systemctl daemon-reload sudo /bin/systemctl enable grafana-server.service
Allow Grafana through firewall
By default, the grafana port is 3000. Adjust as needed.
sudo firewall-cmd --permanent --zone=public --add-port=3000/tcp sudo firewall-cmd --reload
Check Grafana Service Status
sudo systemctl status grafana-server
You may need to start, stop, or restart the service, especially after making changes like we did. The following will start the service:
sudo systemctl start grafana-server
Checking your work
Confirm that both InfluxDB and Grafana are configured for auto-start.
sudo systemctl list-unit-files | grep 'influx\|grafana'
The output from above should look similar to:
grafana-server.service enabled influxd.service enabled influxdb.service enabled
The Grafana Web Interface
The default login is admin admin. Here, you can point Grafana to one or more InfluxDB databases. Later, we can create graphs of performance data through this interface as well.
Keep this web page open for the next step.
Grafana can visualize datapoints from many sources. You can add one or more datasources to a graph. Here we will add one, and make it the default.
In the beginning of this post, we created a database with InfluxDB. In this step, we point Grafana to the database as a datasource. The database is currently empty, but that does not matter.
In the web interface (i.e. http://127.0.0.1:3000) navigate to the Gear icon, and click Datasources. Click add, and then point to what will be our Default database.
Each time we create a new graph or table in Grafana, this datasource will automatically be selected. We can then add more datasources, or change it by clicking Edit on the Chart and changing the datasource.
If you get a
Network Error: Bad Gateway(502) when testing your datasource, scroll up to the HTTP section and click on your address; it will change from gray to white indicating it has been selected. Save and retest and you should be good to go.
Now you can start loading up the database(s) with performance stats from your VMware environment. For this task you can use your own scripts, or my vFlux Stats kit.
About vFlux Stats (optional)
Optionally, download my vFlux Stats Kit from github. These are PowerCLI scripts that connect to VMware vCenter Server and return performance data for virtual machines and ESXi hosts. The data is then written to your InfluxDB database, and served up by Grafana. Currently requires the scripts to be run from a Windows device with PowerShell 3.0 or later (PowerShell 5.1 preferred) and PowerCLI 6.5.1 or later (PowerCLI 10.x preferred).
We can get details for the ESXi host, all virtual machines and even break out a single virtual machine. This example shows a single virtual machine for the past 6 hours.
Q: In the above, can you find an argument to change the layout of the virtual machine (i.e. CPU, Memory, etc.)?
A. Perhaps we can argue that:
- The core count should be decreased (i.e. from 4 vCPU to 2 vCPU)
- Memory can be decreased (unless locked by an app such as SQL)
- When the workload ramps up it primarily uses only 1 vCPU (or does it?)
- If we look closely we can see new threads freely starting on the downswing of other threads that are tearing-down
- We may need need to review the importance of snappy response (decrease core count) vs. the power of long running or multithreaded workloads (stay at 4 vCPU).
- Beyond this, troubleshooting may be subjective and environmental :p
The world is a big place with many things to study and visualize. Perhaps you are analyzing the impact of solar storms and want those datapoints and not just boring servers! Maybe you want an overlay of the two! This is all open source and the creative freedom is yours.
In this write-up we installed and configured
CentOS 7. We created databases on InfluxDB and connected it all together with the beautiful Grafana web interface. The next step is yours, populating datapoints (and bragging)!
Who to Follow
Additional supporting information and examples.
Example - Adding Datasource
Let's look at a datasource being added. In the terminal, we can see that I have started the
influx cli, and issued the
SHOW DATABASES command. In the browser, we can see a working example pointing to the InfluxDB database
About Error 502
Now let's look at a common issue caused by the
URL not being active. This is located in the HTTP section of add Datasources (scroll up).
URL is the InfluxDB address and port. The trick is to click it to activate. If you do not, and try to test the datasource, you will get
Network Error: Bad Gateway(502).
Even though you see your desired InfluxDB address (i.e.
http://localhost:8086, if the text is gray it requires clicking (it's a dropdown box)!
Once you have activated the URL, click
Save and Test. Good to go! Repeat for any other datasources.
A look at the finished datasources, with one selected as default: