Active-Active Server Setup for Tanium 6.2 and Prior

Introduction

The Tanium Server can scale to support a full-size deployment from a single physical or virtual device. To ensure continuous operation in the event of an outage, however, an organization may decide to deploy two or more Tanium Servers configured as an Active/Active Array—each server communicating with the others, each server communicating with Clients, and each server using the same Databases on the same Database Server. Because each server system is actively participating, the workload from reporting clients is distributed among the servers in the array. To provide high-availability should one of the array servers fail, however, each server in the array must meet or exceed the technical requirements to support the entire deployment.

Service Description

  • In an Active/Active Array, each Server reads and writes from the same Databases.
  • Each server creates an entry for itself in the tanium database so that every other server in the Active/Active array knows about it.
  • Each Array Member passes Tanium Messages, such as answers to questions, to its Array Partner(s).
  • Each Array Member is able to host Package Source Files uploaded to one Member. 

Assumptions

The instructions on this page to configure the Tanium application server in an Active/Active array are based on the following assumptions:

  1. The Tanium platform is deployed within your network using one management server to support the total number of computers to be managed.
  2. You have provisioned one or more additional physical or virtual servers according to the same resource recommendations such that any single server in the array can support the total number of computers to be managed.
  3. Each new server can access the Internet directly or through a proxy server to download files from designated domains.
  4. You will be using an existing remote SQL Server instance—dedicated or shared—as the backend for the Array Member servers. The tanium and tanium_archive databases already exist in the SQL Server instance.
  5. You have selected one of the following options to balance incoming traffic from the Tanium Clients across the array member servers:
    Server Selection
    The Server Selection feature of the Windows Tanium Client and the Linux, Unix and MAC Clients version 6.0 or higher enables you to define a list of addresses the client should use to communicate and register with the Tanium Server. The client selects randomly from among the listed server addresses, enabling the system to divide incoming client traffic among the available servers in the Active/Active Array.
    Load Balancer
    You can place the array members behind a network load balancer that will either forward traffic evenly among the application servers or use custom-defined rules to decide which traffic to forward to the Tanium Servers in the array.
    Although configuration of a hardware or software load balancer is beyond the scope of this document, carefully consider the load balancer manufacturer's recommendations as an improperly configured load balancer may decrease Tanium Serve performance.
    Round Robin DNS
    If necessary, Round Robin DNS may be employed if the previous options are not viable. In this configuration, a DNS server will provide two IP addresses for one host entry. Because the Operating System controls name resolution, each Tanium Client will randomly choose one of the entries that resolves when trying to connect.
    In a Round Robin DNS setup, if one Server is removed from the Array for a period of time, some Clients may be unmanageable if they were reporting to the downed server. For this reason, take great care to ensure continuous Tanium Client management; consequently, the Server Selection or Load Balancer approaches are recommended over Round Robin DNS to distribute client communication among the servers in an Active/Active array. Round Robin DNS also tends to be the slowest for agents to fully switch off a down Tanium server because it relies on the OS to replace the cached DNS record with a new, working address.
  6. All Array Members must be running the same version of the software at all times.
  7. All Array Members should be connected to each other via a reliable, high-throughput connection. Messages from all clients flow among all systems, and files are pulled from one server to the other(s).

Implementation Preparation

  1. Contact your Tanium Technical Account Manager to obtain an updated tanium.license file that includes the network address—FQDN, alias, or IP—of all array member servers.
  2. Locate the same version of the Tanium Server installer used to set up the current application server. It is best practice to use the updated and correct license file on all Tanium Servers.
  3. To assist in configuring your Load Balancer for optimal performance, use OS tools to determine the current connection counts on your existing Tanium Server.
  4. Create a new or modify an existing Tanium Server domain service account with the permissions to allow communication between each Array Member Server and the remote database.
  5. Exempt the Tanium Server and Tanium Client processes on the new server from "real-time" or "on-access" scans by AV or other security solutions you may be using. 
  6. Configure any internal or host-based firewalls to allow the following TCP connections:
Source Port Destination
Array Member Server 1 17472 Array Member Server 2-n
Array Member Server 2-n 17472 Array Member Server 1
Array Member Server 1 443 Array Member Server 2-n
Array Member Server 2-n 443 Array Member Server 1
Any computer authorized to run the console 443 Array Member Servers 1-n

Server Implementation and Configuration

  1. On the new server, create the root folder structure for the application server, by default \Program Files\Tanium\Tanium Server
  2. Copy the following files from the installation folder of the existing server to the installation folder you just created on the new server:
    • tanium.pvk
    • tanium.pub
    In addition, for Tanium Server version 6.5.314.x or greater, copy the following files from the installation folder of the original Tanium Server to the installation folder you just created on the new server:
    • SOAPServer.crt
    • SOAPServer.key
  3. Install the Tanium Server software to the new array member server—again, the installer version must match the existing Tanium Server.
  4. Choose "Custom Install" as you work through the installation Wizard.
  5. Choose the database options to use the existing remote SQL Server.
  6. Configure the Tanium Server and Apache services to use the  Tanium Server domain service account. Copy the new tanium.license file that includes the addresses of all array member servers to the installation folder on each server.
  7. Update the Tanium Server registry key on the new server to verify that any missing registry values or differences in registry value data are consistent across all array member servers—in particular, examine any proxy configuration values as well as the registry value data for the TrustedHostList and TrustedCertPath settings on the original server.
  8. Restart the Tanium and Apache Server services on all array member servers.
  9. If you are using a load balancer to distribute client communication among the servers in the Array, update the DNS entry for the Tanium Server and set the IP address to route through the load balancer.

Client Configuration

Regardless of the method you use to balance Tanium Client communication across the array member servers, you must set the Windows client's registry value ServerAddress or the Unix, Linux, Mac client's ServerName value pair in the TaniumClient.ini file to support the option of choice.

Windows Tanium Client
  • Use the /ServerAddress command line argument to set the registry value at initial client installation
  • Use the Tanium Package "Set Tanium Server Name" to modify the registry value for already installed agents.
Unix, Linux, Mac Tanium Client
  • Use the ServerName value pair in the TaniumClient.ini file at initial client installation
  • Create a Tanium Package to modify the existing ServerName value pair in the TaniumClient.ini file for already installed agents.

Server Selection

Tanium Windows Clients

  • To configure Server Selection at initial installation, set the /ServerAddress command line argument equal to a comma separated list of FQDN's, for example:
SetupClient.exe /ServerAddress=srv1.organization.com,srv2.organization.com,srv3.organization.com 
  • To configure Server Selection on clients already installed, use the Tanium Package "Set Tanium Server Name List" to provide the comma separated list of server FQDN's.

Tanium Unix, Linux, Mac Clients

  • Starting with Client version 6.0 Tanium Unix, Linux and Mac clients support Active-Active server selection.

Load Balancer

The ServerAddress registry value or ServerName value pair should resolve to the load balancer's IP address instead of an individual Tanium application server.

As noted in the implementation and configuration section above, be sure to update the DNS entry for the original Tanium application server to point to the IP address of the Load Balancer.

Round Robin DNS

The ServerAddress registry value or ServerName value pair must resolve to the FQDN defined on the DNS server for the Round Robin IP address resolution.

Database Considerations

  • In an Active/Active setup, the Array member Servers share the tanium and tanium_archive databases. The database service is always remote to at least one Server, which typically requires the use of an Active Directory Service Account for the Tanium Server and Apache services on all array member Servers.
  • Ideally, the tanium and tanium_archive databases are remote to the array-member servers, rather than having the Databases on one Server, to maintain continuous operation between reboots the Servers. For proper connectivity to the Database server, ensure both Services on all Servers are using anActive Directory Service Account with appropriate  database privileges.

Download File Synchronization

The process used to reference download files within a Tanium Package determines how each array member server maintains the same Tanium Package files across the respective \Tanium\Tanium Server\Downloads folders:

File added to Tanium Package using "Add local Files..." option

If the console operator uses the "Add local Files..." option to associate files with a package, the files are automatically uploaded to the primary Tanium Server, renamed to match their SHA-256 value and then saved to the \Downloads folder.

  • Within five minutes, the other array member servers will attempt an https connection to the first server to copy any missing uploaded files to the local \Downloads folder.
  • If the array members are using a self-signed SSL certificate and you have not updated the TrustedHostList server registry value on each machine to include the addresses of all array members, this synchronization may fail.

File added to Tanium Package using "Add URI" option

If the console operator uses the "Add URI" option to associate files with a package, the operator references the files by specifying a URI (web URL or \\server\share)

  • The first server will begin downloading the files in a package based on the URI the package creator specified. If the package includes a file's SHA-256 hash value, each remaining array member server will first attempt to download the file from one of the other servers. If the package did not include the file's hash value or the file is not yet available from one of the other array member servers, the respective server downloads the files from the URI source locations defined in the package and saves those files to the local \Downloads folder.
  • If the array member servers must interact with a proxy to download external files and you have not configured the proxy settings, updated web filtering rules, or verified the TrustedHostList or TrustedCertPath settings on each server, the downloads may fail.

In all cases, any Tanium Actions to deploy a package will not execute on targeted endpoints until all array members have received the file. Servers may also exceed 5 minutes to initiate new downloads if the download queue is full.

Tanium Module Server

In an Active-Active server setup both servers will utilize the same module server for the modules that will be installed. At this time, the Tanium modules are not configured to support Active-Active. For this reason, it is common to have an Active-Active pair of Tanium Servers and only one Tanium Module Server.

Importing Tanium Workbenches in an Active-Active Setup

When a Tanium workbench, such as Connect or IOC Detect, is imported from the "Solutions" tab of the console, files are downloaded on the Tanium Server to serve the browser and add the necessary tab in the console. At this time, these files are not sync'd between Tanium Servers. This makes it necessary to import the module on each server in order to have the workbenches available in the console on both servers. This will re-install the module on the module server, but this is a harmless operation.

Since the workbenches in the console appear from left to right in the order that they are imported, it is recommended to import the modules in the same order on TaniumServer01 as you will on TaniumServer02 so that the tabs are in the same order. Contact your Technical Account Manager if you need assistance with re-ordering the workbenches in the console.

There is an existing request being worked on to eliminate the need to import on each server.

Hot Spare Module Server

If the Tanium Module Server goes down, the Tanium console will be crippled in functionality. For this reason, it is an option to have a second Module Server available as a hot spare. If this is a configuration you would like to explore, please contact your Technical Account Manager.

Have more questions? Submit a request