Scalyr Helps Periscope Data Scale Up

Data intelligence provider Periscope Data is using software-as-a-service data visualisation tool Scalyr to help it manage its infrastructure.
“We're aiming to be the most powerful and flexible tool for data analysts,” said Elaine Teoh, VP of Engineering at Periscope Data. “To be successful, we need to be able to see what our system is doing at all times. Scalyr helps us do that.”

Periscope Data is growing fast, apparently doubling revenue and headcount year-on-year. “We need Scalyr to keep up,” said Teoh. As a software-as-a-service offering, staying online and performing well are important to keep that growth going.

“Customers spend 20 or more hours a week using Periscope Data,” said Teoh. “They run their business with us.”

Periscope Data uses Scalyr both to monitor its services and to page someone if things start going awry. “Our infrastructure teams use it the most, but we use a DevOps model, so everyone is on-call some of the time,” said Teoh. The responding engineers then use Scalyr to diagnose the issue.

“Scalyr helped us avoid what would have been a site outage,” said Jeff Watts, a Senior Software Engineer at Periscope Data. “One of the database tables was going to reach a max int value, but Scalyr detected the issue and we migrated to a new table before reaching the limit.” A site outage for a service like Periscope Data would have immediately affected revenues, not to mention shaking the confidence of customers using the site. To say that Periscope Data is pleased with Scalyr is somewhat of an understatement.

Scalyr aims to provide a consolidated view of logs that is also fast and easy to use. Unlike some more general-purpose logging systems, Scalyr is optimised for searching through log data. “Other logging systems are built on key-value stores or document storage, which are fine for storing data, but aren't optimized for log searches,” says Steve Newman, Scalyr Founder and CEO. “You need to do different things with the searches, like providing a summary of all occurrences rather than returning the top ten results,” he says.

Scalyr also aims to be easy-to-use. “With other options, driving the logging system is quite specialised. It can be slow and hard to use without specialist training, so you end up with a gatekeeper model,” said Newman. Scalyr instead tries to provide access to log information to as many people as possible.

The focus on search performance has driven the columnar no-SQL architecture of Scalyr, but also its decision to run as a service rather than a self-hosted system customers run themselves. “We can achieve better economies of scale with a hosted solution,” said Newman.

Scalyr isn't trying to replace all other logging and tracing tools, though. “Scalyr is complementary to other approaches like distributed tracing,” says Newman. Periscope Data also sees value in using different tools for different circumstances. “We mostly use Scalyr for application logging,” said Watts. “Infrastructure logging is done with other tools, such as Cloudwatch for RDS databases.” Periscope Data also uses free tools bundled with different components of the software stack where they work well enough, supplementing them with Scalyr when something more is needed.

There are a surprising number of log management and application performance management tools out there, and many readers will be familiar with the likes of Splunk, Sumo Logic, and Datadog. Maybe add Scalyr to your list of tools to test if you're in the market for a new option.

This article originally misspelled Scalyr because apparently I can't type, and my fingers typed Peter instead of Jeff. I didn't check it carefully enough, so I'll try harder from now on.

This article first appeared in Forbes.com here.