Installing Elasticsearch

Elasticsearch is provided in the following package formats:

zip/tar.gz

The zip and tar.gz packages are suitable for installation on any system and are the easiest choice for getting started with Elasticsearch.

Install Elasticsearch with .zip or .tar.gz or Install Elasticsearch on Windows

deb

The deb package is suitable for Debian, Ubuntu, and other Debian-based systems. Debian packages may be downloaded from the Elasticsearch website or from our Debian repository.

Install Elasticsearch with Debian Package

rpm

The rpm package is suitable for installation on Red Hat, Centos, SLES, OpenSuSE and other RPM-based systems. RPMs may be downloaded from the Elasticsearch website or from our RPM repository.

Install Elasticsearch with RPM

docker

An image is available for running Elasticsearch as a Docker container. It ships with X-Pack pre-installed and may be downloaded from the Elastic Docker Registry.

Install Elasticsearch with Docker

Configuration Management Tools

We also provide the following configuration management tools to help with large deployments:

Install Elasticsearch with .zip or .tar.gz

Elasticsearch is provided as a .zip and as a .tar.gz package. These packages can be used to install Elasticsearch on any system and are the easiest package format to use when trying out Elasticsearch.

The latest stable version of Elasticsearch can be found on the Download Elasticsearch page. Other versions can be found on the Past Releases page.

Note

Elasticsearch requires Java 8 or later. Use the official Oracle distribution or an open-source distribution such as OpenJDK.

Download and install the .zip package

The .zip archive for Elasticsearch v5.1.1 can be downloaded and installed as follows:

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.1.1.zip
sha1sum elasticsearch-5.1.1.zip 
unzip elasticsearch-5.1.1.zip
cd elasticsearch-5.1.1/ 

Compare the SHA produced by sha1sum or shasum with the published SHA.

This directory is known as $ES_HOME.

Download and install the .tar.gz package

The .tar.gz archive for Elasticsearch v5.1.1 can be downloaded and installed as follows:

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.1.1.tar.gz
sha1sum elasticsearch-5.1.1.tar.gz 
tar -xzf elasticsearch-5.1.1.tar.gz
cd elasticsearch-5.1.1/ 

Compare the SHA produced by sha1sum or shasum with the published SHA.

This directory is known as $ES_HOME.

Running Elasticsearch from the command line

Elasticsearch can be started from the command line as follows:

./bin/elasticsearch

By default, Elasticsearch runs in the foreground, prints its logs to the standard output (stdout), and can be stopped by pressing Ctrl-C.

Checking that Elasticsearch is running

You can test that your Elasticsearch node is running by sending an HTTP request to port 9200 on localhost:

GET /

which should give you a response something like this:

{
  "name" : "Cp8oag6",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "AT69_T_DTp-1qgIJlatQqA",
  "version" : {
    "number" : "5.1.1",
    "build_hash" : "f27399d",
    "build_date" : "2016-03-30T09:51:41.449Z",
    "build_snapshot" : false,
    "lucene_version" : "6.3.0"
  },
  "tagline" : "You Know, for Search"
}

Log printing to stdout can be disabled using the -q or --quiet option on the command line.

Running as a daemon

To run Elasticsearch as a daemon, specify -d on the command line, and record the process ID in a file using the -p option:

./bin/elasticsearch -d -p pid

Log messages can be found in the $ES_HOME/logs/ directory.

To shut down Elasticsearch, kill the process ID recorded in the pid file:

kill `cat pid`
Note

The startup scripts provided in the RPM and Debian packages take care of starting and stopping the Elasticsearch process for you.

Configuring Elasticsearch on the command line

Elasticsearch loads its configuration from the $ES_HOME/config/elasticsearch.yml file by default. The format of this config file is explained in Configuring Elasticsearch.

Any settings that can be specified in the config file can also be specified on the command line, using the -E syntax as follows:

./bin/elasticsearch -d -Ecluster.name=my_cluster -Enode.name=node_1
Tip

Typically, any cluster-wide settings (like cluster.name) should be added to the elasticsearch.yml config file, while any node-specific settings such as node.name could be specified on the command line.

Directory layout of .zip and .tar.gz archives

The .zip and .tar.gz packages are entirely self-contained. All files and directories are, by default, contained within $ES_HOME — the directory created when unpacking the archive.

This is very convenient because you don’t have to create any directories to start using Elasticsearch, and uninstalling Elasticsearch is as easy as removing the $ES_HOME directory. However, it is advisable to change the default locations of the config directory, the data directory, and the logs directory so that you do not delete important data later on.

Type Description Default Location Setting

home

Elasticsearch home directory or $ES_HOME

Directory created by unpacking the archive

bin

Binary scripts including elasticsearch to start a node and elasticsearch-plugin to install plugins

$ES_HOME/bin

conf

Configuration files including elasticsearch.yml

$ES_HOME/config

path.conf

data

The location of the data files of each index / shard allocated on the node. Can hold multiple locations.

$ES_HOME/data

path.data

logs

Log files location.

$ES_HOME/logs

path.logs

plugins

Plugin files location. Each plugin will be contained in a subdirectory.

$ES_HOME/plugins

repo

Shared file system repository locations. Can hold multiple locations. A file system repository can be placed in to any subdirectory of any directory specified here.

Not configured

path.repo

script

Location of script files.

$ES_HOME/scripts

path.scripts

Next steps

You now have a test Elasticsearch environment set up. Before you start serious development or go into production with Elasticsearch, you will need to do some additional setup:

Install Elasticsearch with Debian Package

The Debian package for Elasticsearch can be downloaded from our website or from our APT repository. It can be used to install Elasticsearch on any Debian-based system such as Debian and Ubuntu.

The latest stable version of Elasticsearch can be found on the Download Elasticsearch page. Other versions can be found on the Past Releases page.

Note

Elasticsearch requires Java 8 or later. Use the official Oracle distribution or an open-source distribution such as OpenJDK.

Import the Elasticsearch PGP Key

We sign all of our packages with the Elasticsearch Signing Key (PGP key D88E42B4, available from https://pgp.mit.edu) with fingerprint:

4609 5ACC 8548 582C 1A26 99A9 D27D 666C D88E 42B4

Download and install the public signing key:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

Installing from the APT repository

You may need to install the apt-transport-https package on Debian before proceeding:

sudo apt-get install apt-transport-https

Save the repository definition to /etc/apt/sources.list.d/elastic-5.x.list:

echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list
Warning

Do not use add-apt-repository as it will add a deb-src entry as well, but we do not provide a source package. If you have added the deb-src entry, you will see an error like the following:

Unable to find expected entry 'main/source/Sources' in Release file
(Wrong sources.list entry or malformed file)

Delete the deb-src entry from the /etc/apt/sources.list file and the installation should work as expected.

You can install the Elasticsearch Debian package with:

sudo apt-get update && sudo apt-get install elasticsearch
Warning

If two entries exist for the same Elasticsearch repository, you will see an error like this during apt-get update:

Duplicate sources.list entry https://artifacts.elastic.co/packages/5.x/apt/ ...`

Examine /etc/apt/sources.list.d/elasticsearch-5.x.list for the duplicate entry or locate the duplicate entry amongst the files in /etc/apt/sources.list.d/ and the /etc/apt/sources.list file.

Note

On systemd-based distributions, the installation scripts will attempt to set kernel parameters (e.g., vm.max_map_count); you can skip this by setting the environment variable ES_SKIP_SET_KERNEL_PARAMETERS to true.

Download and install the Debian package manually

The Debian package for Elastisearch v5.1.1 can be downloaded from the website and installed as follows:

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.1.1.deb
sha1sum elasticsearch-5.1.1.deb 
sudo dpkg -i elasticsearch-5.1.1.deb

Compare the SHA produced by sha1sum or shasum with the published SHA.

SysV init vs systemd

Elasticsearch is not started automatically after installation. How to start and stop Elasticsearch depends on whether your system uses SysV init or systemd (used by newer distributions). You can tell which is being used by running this command:

ps -p 1

Running Elasticsearch with SysV init

Use the update-rc.d command to configure Elasticsearch to start automatically when the system boots up:

sudo update-rc.d elasticsearch defaults 95 10

Elasticsearch can be started and stopped using the service command:

sudo -i service elasticsearch start
sudo -i service elasticsearch stop

If Elasticsearch fails to start for any reason, it will print the reason for failure to STDOUT. Log files can be found in /var/log/elasticsearch/.

Running Elasticsearch with systemd

To configure Elasticsearch to start automatically when the system boots up, run the following commands:

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service

Elasticsearch can be started and stopped as follows:

sudo systemctl start elasticsearch.service
sudo systemctl stop elasticsearch.service

These commands provide no feedback as to whether Elasticsearch was started successfully or not. Instead, this information will be written in the log files located in /var/log/elasticsearch/.

By default the Elasticsearch service doesn’t log information in the systemd journal. To enable journalctl logging, the --quiet option must be removed from the ExecStart command line in the elasticsearch.service file.

When systemd logging is enabled, the logging information are available using the journalctl commands:

To tail the journal:

sudo journalctl -f

To list journal entries for the elasticsearch service:

sudo journalctl --unit elasticsearch

To list journal entries for the elasticsearch service starting from a given time:

sudo journalctl --unit elasticsearch --since  "2016-10-30 18:17:16"

Check man journalctl or https://www.freedesktop.org/software/systemd/man/journalctl.html for more command line options.

Checking that Elasticsearch is running

You can test that your Elasticsearch node is running by sending an HTTP request to port 9200 on localhost:

GET /

which should give you a response something like this:

{
  "name" : "Cp8oag6",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "AT69_T_DTp-1qgIJlatQqA",
  "version" : {
    "number" : "5.1.1",
    "build_hash" : "f27399d",
    "build_date" : "2016-03-30T09:51:41.449Z",
    "build_snapshot" : false,
    "lucene_version" : "6.3.0"
  },
  "tagline" : "You Know, for Search"
}

Configuring Elasticsearch

Elasticsearch loads its configuration from the /etc/elasticsearch/elasticsearch.yml file by default. The format of this config file is explained in Configuring Elasticsearch.

The Debian package also has a system configuration file (/etc/default/elasticsearch), which allows you to set the following parameters:

ES_USER

The user to run as, defaults to elasticsearch.

ES_GROUP

The group to run as, defaults to elasticsearch.

JAVA_HOME

Set a custom Java path to be used.

MAX_OPEN_FILES

Maximum number of open files, defaults to 65536.

MAX_LOCKED_MEMORY

Maximum locked memory size. Set to unlimited if you use the bootstrap.memory_lock option in elasticsearch.yml.

MAX_MAP_COUNT

Maximum number of memory map areas a process may have. If you use mmapfs as index store type, make sure this is set to a high value. For more information, check the linux kernel documentation about max_map_count. This is set via sysctl before starting elasticsearch. Defaults to 262144.

LOG_DIR

Log directory, defaults to /var/log/elasticsearch.

DATA_DIR

Data directory, defaults to /var/lib/elasticsearch.

CONF_DIR

Configuration file directory (which needs to include elasticsearch.yml and log4j2.properties files), defaults to /etc/elasticsearch.

ES_JAVA_OPTS

Any additional JVM system properties you may want to apply.

RESTART_ON_UPGRADE

Configure restart on package upgrade, defaults to false. This means you will have to restart your elasticsearch instance after installing a package manually. The reason for this is to ensure, that upgrades in a cluster do not result in a continuous shard reallocation resulting in high network traffic and reducing the response times of your cluster.

Note

Distributions that use systemd require that system resource limits be configured via systemd rather than via the /etc/sysconfig/elasticsearch file. See Systemd configuration for more information.

Directory layout of Debian package

The Debian package places config files, logs, and the data directory in the appropriate locations for a Debian-based system:

Type Description Default Location Setting

home

Elasticsearch home directory or $ES_HOME

/usr/share/elasticsearch

bin

Binary scripts including elasticsearch to start a node and elasticsearch-plugin to install plugins

/usr/share/elasticsearch/bin

conf

Configuration files including elasticsearch.yml

/etc/elasticsearch

path.conf

conf

Environment variables including heap size, file descriptors.

/etc/default/elasticsearch

data

The location of the data files of each index / shard allocated on the node. Can hold multiple locations.

/var/lib/elasticsearch

path.data

logs

Log files location.

/var/log/elasticsearch

path.logs

plugins

Plugin files location. Each plugin will be contained in a subdirectory.

/usr/share/elasticsearch/plugins

repo

Shared file system repository locations. Can hold multiple locations. A file system repository can be placed in to any subdirectory of any directory specified here.

Not configured

path.repo

script

Location of script files.

/etc/elasticsearch/scripts

path.scripts

Next steps

You now have a test Elasticsearch environment set up. Before you start serious development or go into production with Elasticsearch, you will need to do some additional setup:

Install Elasticsearch with RPM

The RPM for Elasticsearch can be downloaded from our website or from our RPM repository. It can be used to install Elasticsearch on any RPM-based system such as OpenSuSE, SLES, Centos, Red Hat, and Oracle Enterprise.

Note

RPM install is not supported on distributions with old versions of RPM, such as SLES 11 and CentOS 5. Please see Install Elasticsearch with .zip or .tar.gz instead.

The latest stable version of Elasticsearch can be found on the Download Elasticsearch page. Other versions can be found on the Past Releases page.

Note

Elasticsearch requires Java 8 or later. Use the official Oracle distribution or an open-source distribution such as OpenJDK.

Import the Elasticsearch PGP Key

We sign all of our packages with the Elasticsearch Signing Key (PGP key D88E42B4, available from https://pgp.mit.edu) with fingerprint:

4609 5ACC 8548 582C 1A26 99A9 D27D 666C D88E 42B4

Download and install the public signing key:

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Installing from the RPM repository

Create a file called elasticsearch.repo in the /etc/yum.repos.d/ directory for RedHat based distributions, or in the /etc/zypp/repos.d/ directory for OpenSuSE based distributions, containing:

[elasticsearch-5.x]
name=Elasticsearch repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

And your repository is ready for use. You can now install Elasticsearch with one of the following commands:

sudo yum install elasticsearch 
sudo dnf install elasticsearch 
sudo zypper install elasticsearch 

Use yum on CentOS and older Red Hat based distributions.

Use dnf on Fedora and other newer Red Hat distributions.

Use zypper on OpenSUSE based distributions

Download and install the RPM manually

The RPM for Elastisearch v5.1.1 can be downloaded from the website and installed as follows:

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.1.1.rpm
sha1sum elasticsearch-5.1.1.rpm 
sudo rpm --install elasticsearch-5.1.1.rpm

Compare the SHA produced by sha1sum or shasum with the published SHA.

Note

On systemd-based distributions, the installation scripts will attempt to set kernel parameters (e.g., vm.max_map_count); you can skip this by setting the environment variable ES_SKIP_SET_KERNEL_PARAMETERS to true.

SysV init vs systemd

Elasticsearch is not started automatically after installation. How to start and stop Elasticsearch depends on whether your system uses SysV init or systemd (used by newer distributions). You can tell which is being used by running this command:

ps -p 1

Running Elasticsearch with SysV init

Use the chkconfig command to configure Elasticsearch to start automatically when the system boots up:

sudo chkconfig --add elasticsearch

Elasticsearch can be started and stopped using the service command:

sudo -i service elasticsearch start
sudo -i service elasticsearch stop

If Elasticsearch fails to start for any reason, it will print the reason for failure to STDOUT. Log files can be found in /var/log/elasticsearch/.

Running Elasticsearch with systemd

To configure Elasticsearch to start automatically when the system boots up, run the following commands:

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service

Elasticsearch can be started and stopped as follows:

sudo systemctl start elasticsearch.service
sudo systemctl stop elasticsearch.service

These commands provide no feedback as to whether Elasticsearch was started successfully or not. Instead, this information will be written in the log files located in /var/log/elasticsearch/.

By default the Elasticsearch service doesn’t log information in the systemd journal. To enable journalctl logging, the --quiet option must be removed from the ExecStart command line in the elasticsearch.service file.

When systemd logging is enabled, the logging information are available using the journalctl commands:

To tail the journal:

sudo journalctl -f

To list journal entries for the elasticsearch service:

sudo journalctl --unit elasticsearch

To list journal entries for the elasticsearch service starting from a given time:

sudo journalctl --unit elasticsearch --since  "2016-10-30 18:17:16"

Check man journalctl or https://www.freedesktop.org/software/systemd/man/journalctl.html for more command line options.

Checking that Elasticsearch is running

You can test that your Elasticsearch node is running by sending an HTTP request to port 9200 on localhost:

GET /

which should give you a response something like this:

{
  "name" : "Cp8oag6",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "AT69_T_DTp-1qgIJlatQqA",
  "version" : {
    "number" : "5.1.1",
    "build_hash" : "f27399d",
    "build_date" : "2016-03-30T09:51:41.449Z",
    "build_snapshot" : false,
    "lucene_version" : "6.3.0"
  },
  "tagline" : "You Know, for Search"
}

Configuring Elasticsearch

Elasticsearch loads its configuration from the /etc/elasticsearch/elasticsearch.yml file by default. The format of this config file is explained in Configuring Elasticsearch.

The RPM also has a system configuration file (/etc/sysconfig/elasticsearch), which allows you to set the following parameters:

ES_USER

The user to run as, defaults to elasticsearch.

ES_GROUP

The group to run as, defaults to elasticsearch.

JAVA_HOME

Set a custom Java path to be used.

MAX_OPEN_FILES

Maximum number of open files, defaults to 65536.

MAX_LOCKED_MEMORY

Maximum locked memory size. Set to unlimited if you use the bootstrap.memory_lock option in elasticsearch.yml.

MAX_MAP_COUNT

Maximum number of memory map areas a process may have. If you use mmapfs as index store type, make sure this is set to a high value. For more information, check the linux kernel documentation about max_map_count. This is set via sysctl before starting elasticsearch. Defaults to 262144.

LOG_DIR

Log directory, defaults to /var/log/elasticsearch.

DATA_DIR

Data directory, defaults to /var/lib/elasticsearch.

CONF_DIR

Configuration file directory (which needs to include elasticsearch.yml and log4j2.properties files), defaults to /etc/elasticsearch.

ES_JAVA_OPTS

Any additional JVM system properties you may want to apply.

RESTART_ON_UPGRADE

Configure restart on package upgrade, defaults to false. This means you will have to restart your elasticsearch instance after installing a package manually. The reason for this is to ensure, that upgrades in a cluster do not result in a continuous shard reallocation resulting in high network traffic and reducing the response times of your cluster.

Note

Distributions that use systemd require that system resource limits be configured via systemd rather than via the /etc/sysconfig/elasticsearch file. See Systemd configuration for more information.

Directory layout of RPM

The RPM places config files, logs, and the data directory in the appropriate locations for an RPM-based system:

Type Description Default Location Setting

home

Elasticsearch home directory or $ES_HOME

/usr/share/elasticsearch

bin

Binary scripts including elasticsearch to start a node and elasticsearch-plugin to install plugins

/usr/share/elasticsearch/bin

conf

Configuration files including elasticsearch.yml

/etc/elasticsearch

path.conf

conf

Environment variables including heap size, file descriptors.

/etc/sysconfig/elasticsearch

data

The location of the data files of each index / shard allocated on the node. Can hold multiple locations.

/var/lib/elasticsearch

path.data

logs

Log files location.

/var/log/elasticsearch

path.logs

plugins

Plugin files location. Each plugin will be contained in a subdirectory.

/usr/share/elasticsearch/plugins

repo

Shared file system repository locations. Can hold multiple locations. A file system repository can be placed in to any subdirectory of any directory specified here.

Not configured

path.repo

script

Location of script files.

/etc/elasticsearch/scripts

path.scripts

Next steps

You now have a test Elasticsearch environment set up. Before you start serious development or go into production with Elasticsearch, you will need to do some additional setup:

Install Elasticsearch on Windows

Elasticsearch can be installed on Windows using the .zip package. This comes with a elasticsearch-service.bat command which will setup Elasticsearch to run as a service.

The latest stable version of Elasticsearch can be found on the Download Elasticsearch page. Other versions can be found on the Past Releases page.

Note

Elasticsearch requires Java 8 or later. Use the official Oracle distribution or an open-source distribution such as OpenJDK.

Download and install the .zip package

Download the .zip archive for Elasticsearch v5.1.1 from: https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.1.1.zip

Unzip it with your favourite unzip tool. This will create a folder called elasticsearch-5.1.1, which we will refer to as %ES_HOME%. In a terminal window, cd to the %ES_HOME% directory, for instance:

cd c:\elasticsearch-5.1.1

Running Elasticsearch from the command line

Elasticsearch can be started from the command line as follows:

.\bin\elasticsearch

By default, Elasticsearch runs in the foreground, prints its logs to STDOUT, and can be stopped by pressing Ctrl-C.

Configuring Elasticsearch on the command line

Elasticsearch loads its configuration from the %ES_HOME%/config/elasticsearch.yml file by default. The format of this config file is explained in Configuring Elasticsearch.

Any settings that can be specified in the config file can also be specified on the command line, using the -E syntax as follows:

./bin/elasticsearch -Ecluster.name=my_cluster -Enode.name=node_1
Note

Values that contain spaces must be surrounded with quotes. For instance -Epath.logs="C:\My Logs\logs".

Tip

Typically, any cluster-wide settings (like cluster.name) should be added to the elasticsearch.yml config file, while any node-specific settings such as node.name could be specified on the command line.

Checking that Elasticsearch is running

You can test that your Elasticsearch node is running by sending an HTTP request to port 9200 on localhost:

GET /

which should give you a response something like this:

{
  "name" : "Cp8oag6",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "AT69_T_DTp-1qgIJlatQqA",
  "version" : {
    "number" : "5.1.1",
    "build_hash" : "f27399d",
    "build_date" : "2016-03-30T09:51:41.449Z",
    "build_snapshot" : false,
    "lucene_version" : "6.3.0"
  },
  "tagline" : "You Know, for Search"
}

Installing Elasticsearch as a Service on Windows

Elasticsearch can be installed as a service to run in the background or start automatically at boot time without any user interaction. This can be achieved through the elasticsearch-service.bat script in the bin\ folder which allows one to install, remove, manage or configure the service and potentially start and stop the service, all from the command-line.

c:\elasticsearch-5.1.1\bin>elasticsearch-service

Usage: elasticsearch-service.bat install|remove|start|stop|manager [SERVICE_ID]

The script requires one parameter (the command to execute) followed by an optional one indicating the service id (useful when installing multiple Elasticsearch services).

The commands available are:

install

Install Elasticsearch as a service

remove

Remove the installed Elasticsearch service (and stop the service if started)

start

Start the Elasticsearch service (if installed)

stop

Stop the Elasticsearch service (if started)

manager

Start a GUI for managing the installed service

Based on the architecture of the available JDK/JRE (set through JAVA_HOME), the appropriate 64-bit(x64) or 32-bit(x86) service will be installed. This information is made available during install:

c:\elasticsearch-5.1.1\bin>elasticsearch-service install
Installing service      :  "elasticsearch-service-x64"
Using JAVA_HOME (64-bit):  "c:\jvm\jdk1.8"
The service 'elasticsearch-service-x64' has been installed.
Note

While a JRE can be used for the Elasticsearch service, due to its use of a client VM (as opposed to a server JVM which offers better performance for long-running applications) its usage is discouraged and a warning will be issued.

Note

Upgrading (or downgrading) JVM versions does not require the service to be reinstalled. However, upgrading across JVM types (e.g. JRE versus SE) is not supported, and does require the service to be reinstalled.

Customizing service settings

The Elasticsearch service can be configured prior to installation by setting the the following environment variables (either using the set command from the command line, or through the System Properties->Environment Variables GUI).

SERVICE_ID

A unique identifier for the service. Useful if installing multiple instances on the same machine. Defaults to elasticsearch-service-x86 (on 32-bit Windows) or elasticsearch-service-x64 (on 64-bit Windows).

SERVICE_USERNAME

The user to run as, defaults to the local system account.

SERVICE_PASSWORD

The password for the user specified in %SERVICE_USERNAME%.

SERVICE_DISPLAY_NAME

The name of the service. Defaults to Elasticsearch <version> %SERVICE_ID%.

SERVICE_DESCRIPTION

The description of the service. Defaults to Elasticsearch <version> Windows Service - https://elastic.co.

JAVA_HOME

The installation directory of the desired JVM to run the service under.

LOG_DIR

Log directory, defaults to %ES_HOME%\logs.

DATA_DIR

Data directory, defaults to %ES_HOME%\data.

CONF_DIR

Configuration file directory (which needs to include elasticsearch.yml and log4j2.properties files), defaults to %ES_HOME%\conf.

ES_JAVA_OPTS

Any additional JVM system properties you may want to apply.

ES_START_TYPE

Startup mode for the service. Can be either auto or manual (default).

ES_STOP_TIMEOUT

The timeout in seconds that procrun waits for service to exit gracefully. Defaults to 0.

Note

At its core, elasticsearch-service.bat relies on Apache Commons Daemon project to install the service. Environment variables set prior to the service installation are copied and will be used during the service lifecycle. This means any changes made to them after the installation will not be picked up unless the service is reinstalled.

Note

On Windows, the heap size can be configured as for any other Elasticsearch installation when running Elasticsearch from the command line, or when installing Elasticsearch as a service for the first time. To adjust the heap size for an already installed service, use the service manager: bin\elasticsearch-service.bat manager.

Using the Manager GUI
It is also possible to configure the service after it’s been installed using the manager GUI (elasticsearch-service-mgr.exe), which offers insight into the installed service, including its status, startup type, JVM, start and stop settings amongst other things. Simply invoking elasticsearch-service.bat manager from the command-line will open up the manager window:
Windows Service Manager GUI

Most changes (like JVM settings) made through the manager GUI will require a restart of the service in order to take affect.

Directory layout of .zip archive

The .zip package is entirely self-contained. All files and directories are, by default, contained within %ES_HOME% — the directory created when unpacking the archive.

This is very convenient because you don’t have to create any directories to start using Elasticsearch, and uninstalling Elasticsearch is as easy as removing the %ES_HOME% directory. However, it is advisable to change the default locations of the config directory, the data directory, and the logs directory so that you do not delete important data later on.

Type Description Default Location Setting

home

Elasticsearch home directory or %ES_HOME%

Directory created by unpacking the archive

bin

Binary scripts including elasticsearch to start a node and elasticsearch-plugin to install plugins

%ES_HOME%\bin

conf

Configuration files including elasticsearch.yml

%ES_HOME%\config

path.conf

data

The location of the data files of each index / shard allocated on the node. Can hold multiple locations.

%ES_HOME%\data

path.data

logs

Log files location.

%ES_HOME%\logs

path.logs

plugins

Plugin files location. Each plugin will be contained in a subdirectory.

%ES_HOME%\plugins

repo

Shared file system repository locations. Can hold multiple locations. A file system repository can be placed in to any subdirectory of any directory specified here.

Not configured

path.repo

script

Location of script files.

%ES_HOME%\scripts

path.scripts

Next steps

You now have a test Elasticsearch environment set up. Before you start serious development or go into production with Elasticsearch, you will need to do some additional setup:

Install Elasticsearch with Docker

Elasticsearch is also available as a Docker image. The image is built with X-Pack.

Security note

Note

X-Pack is preinstalled in this image. Please take a few minutes to familiarize yourself with X-Pack Security and how to change default passwords. The default password for the elastic user is changeme.

Note

X-Pack includes a trial license for 30 days. After that, you can obtain one of the available subscriptions or disable Security. The Basic license is free and includes the Monitoring extension.

Obtaining Elasticsearch for Docker is as simple as issuing a docker pull command against the Elastic Docker registry.

The Docker image can be retrieved with the following command:

docker pull docker.elastic.co/elasticsearch/elasticsearch:5.1.1

Running Elasticsearch from the command line

Development mode

Elasticsearch can be quickly started for development or testing use with the following command:

docker run -p 9200:9200 -e "http.host=0.0.0.0" -e "transport.host=127.0.0.1" docker.elastic.co/elasticsearch/elasticsearch:5.1.1

Production mode

Important

The vm_max_map_count kernel setting needs to be set to at least 262144 for production use. Depending on your platform:

  • Linux

    The vm_map_max_count setting should be set permanently in /etc/sysctl.conf:

    $ grep vm.max_map_count /etc/sysctl.conf
    vm.max_map_count=262144

    To apply the setting on a live system type: sysctl -w vm.max_map_count=262144

  • OSX with Docker for Mac

    The vm_max_map_count setting must be set within the xhyve virtual machine:

    $ screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty

    Log in with root and no password. Then configure the sysctl setting as you would for Linux:

    sysctl -w vm.max_map_count=262144
  • OSX with Docker Toolbox

    The vm_max_map_count setting must be set via docker-machine:

    docker-machine ssh
    sudo sysctl -w vm.max_map_count=262144

The following example brings up a cluster comprising two Elasticsearch nodes. To bring up the cluster, use the docker-compose.yml and just type:

docker-compose up
Note

docker-compose is not pre-installed with Docker on Linux. Instructions for installing it can be found on the docker-compose webpage.

The node elasticsearch1 listens on localhost:9200 while elasticsearch2 talks to elasticsearch1 over a Docker network.

This example also uses Docker named volumes, called esdata1 and esdata2 which will be created if not already present.

docker-compose.yml:

version: '2'
services:
  elasticsearch1:
    image: docker.elastic.co/elasticsearch/elasticsearch:5.1.1
    container_name: elasticsearch1
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    mem_limit: 1g
    cap_add:
      - IPC_LOCK
    volumes:
      - esdata1:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    networks:
      - esnet
  elasticsearch2:
    image: docker.elastic.co/elasticsearch/elasticsearch:5.1.1
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - "discovery.zen.ping.unicast.hosts=elasticsearch1"
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    mem_limit: 1g
    cap_add:
      - IPC_LOCK
    volumes:
      - esdata2:/usr/share/elasticsearch/data
    networks:
      - esnet

volumes:
  esdata1:
    driver: local
  esdata2:
    driver: local

networks:
  esnet:
    driver: bridge

To stop the cluster, type docker-compose down. Data volumes will persist, so it’s possible to start the cluster again with the same data using docker-compose up. To destroy the cluster and the data volumes just type docker-compose down -v.

Inspect status of cluster:

curl -u elastic http://127.0.0.1:9200/_cat/health
Enter host password for user 'elastic':
1472225929 15:38:49 docker-cluster green 2 2 4 2 0 0 0 0 - 100.0%

Log messages go to the console and are handled by the configured Docker logging driver. By default you can access logs with docker logs.

Configuring Elasticsearch with Docker

Elasticsearch loads its configuration from files under /usr/share/elasticsearch/config/. These configuration files are documented in Configuring Elasticsearch and Setting JVM options.

The image offers several methods for configuring Elasticsearch settings with the conventional approach being to provide customized files, i.e. elasticsearch.yml, but it’s also possible to use environment variables to set options:

A. Present the parameters via Docker environment variables

For example, to define the cluster name with docker run you can pass -e "cluster.name=mynewclustername". Double quotes are required.

Note

There is a difference between defining default settings and normal settings. The former are prefixed with default. and cannot override normal settings, if defined.

B. Bind-mounted configuration

Create your custom config file and mount this over the image’s corresponding file. For example, bind-mounting a custom_elasticsearch.yml with docker run can be accomplished with the parameter:

-v full_path_to/custom_elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
Important

custom_elasticsearch.yml should be readable by uid:gid 1000:1000

C. Customized image

In some environments, it may make more sense to prepare a custom image containing your configuration. A Dockerfile to achieve this may be as simple as:

FROM docker.elastic.co/elasticsearch/elasticsearch:5.1.1
ADD elasticsearch.yml /usr/share/elasticsearch/config/
USER root
chown elasticsearch:elasticsearch config/elasticsearch.yml
USER elasticsearch

You could then build and try the image with something like:

docker build --tag=elasticsearch-custom .
docker run -ti -v /usr/share/elasticsearch/data elasticsearch-custom

D. Override the image’s default CMD

Options can be passed as command-line options to the Elasticsearch process by overriding the default command for the image. For example:

docker run <various parameters> bin/elasticsearch -Ecluster.name=mynewclustername

Notes for production use and defaults

We have collected a number of best practices for production use.

Note

Any Docker parameters mentioned below assume the use of docker run.

  1. It is important to correctly set capabilities and ulimits via the Docker CLI. As seen earlier in the example docker-compose.yml, the following options are required:

    --cap-add=IPC_LOCK --ulimit memlock=-1:-1 --ulimit nofile=65536:65536
  2. Ensure bootstrap.memory_lock is set to true as explained in "Disable swapping".

    This can be achieved through any of the configuration methods, e.g. by setting the appropriate environments variable with -e "bootstrap.memory_lock=true".

  3. The image exposes TCP ports 9200 and 9300. For clusters it is recommended to randomize the published ports with --publish-all, unless you are pinning one container per host.
  4. Use the ES_JAVA_OPTS environment variable to set heap size, e.g. to use 16GB use -e ES_JAVA_OPTS="-Xms16g -Xmx16g" with docker run. It is also recommended to set a memory limit for the container.
  5. Pin your deployments to a specific version of the Elasticsearch Docker image, e.g. docker.elastic.co/elasticsearch/elasticsearch:5.1.1.
  6. Always use a volume bound on /usr/share/elasticsearch/data, as shown in the production example, for the following reasons:

    1. The data of your elasticsearch node won’t be lost if the container is killed
    2. Elasticsearch is I/O sensitive and the Docker storage driver is not ideal for fast I/O
    3. It allows the use of advanced Docker volume plugins
  7. If you are using the devicemapper storage driver (default on at least RedHat (rpm) based distributions) make sure you are not using the default loop-lvm mode. Configure docker-engine to use direct-lvm instead.
  8. Consider centralizing your logs by using a different logging driver. Also note that the default json-file logging driver is not ideally suited for production use.

Next steps

You now have a test Elasticsearch environment set up. Before you start serious development or go into production with Elasticsearch, you will need to do some additional setup: