Elasticsearch vs OpenSearch

In this post, we are going to compare Elasticsearch and OpenSearch, beginning with some history, followed by a high-level comparison of the two.

History

Essentially OpenSearch originates from a dispute between Amazon Web Services (AWS) and the company behind Elasticsearch, Elastic. Let’s begin by talking about the history of all of this. This is a topic of different opinions and interests, so I will try to be as neutral as possible.

Open Source and X-Pack

Elasticsearch was initially released as an open source project under the Apache 2.0 license. I am by no means a license expert, but essentially this meant that the project was open source and that the software could be freely used, distributed, and even modified. In other words, it was released under an extremely permissible license with minimal restrictions. Since Elastic is a commercial company, they began developing a set of commercial features besides the freely available ones. These features were developed under a more restrictive license. The source code was still publicly available, but a paid license was required to use these commercial features. This set of features was called X-Pack at the time and included things such as security features, monitoring, alerting, and machine learning. The Elastic company was largely operating on venture capital at the time, so X-Pack was a way for the company to begin generating revenue to fund its operations.

Amazon Elasticsearch Service

In 2015, AWS launched a new service for running managed Elasticsearch clusters, which they named Amazon Elasticsearch Service. Elastic was not pleased with this at all, in part because the name violated their trademark of “Elasticsearch.” This, they believed, indicated that the service was offered as a partnership between Elastic and AWS, which was not the case. Not long after, Elastic released their own managed service named Elastic Cloud. As you can probably imagine, the AWS service was now a direct competitor. Users could still enable X-Pack features on AWS if they purchased a license from Elastic, but otherwise Elastic would not see any revenue from AWS. Needless to say, relations between the two companies were not great at this point.

Open Distro for Elasticsearch

Fast forward to March 2019, AWS released something called Open Distro for Elasticsearch as a direct response to the proprietary features that Elastic had been incorporating into Elasticsearch and Kibana. While the core of the technologies were still under an Apache 2.0 license, AWS felt that the line between the open source and commercial parts was becoming increasingly blurry. They therefore released a distribution of Elasticsearch and Kibana under the Apache 2.0 license that included many of the same features that Elastic had built into X-Pack. Now users could use at least similar features free of charge. To be clear, Open Distro for Elasticsearch was not a fork, but a distribution with added features. Also worth noting is that a few months after the release, Elastic made some X-Pack features available for free, including basic security and monitoring features. This was largely a response to AWS’ argument that security should not be a commercial feature, as well as pressure from the community.

Anyway, you probably won’t be surprised to hear that Elastic wasn’t too happy about Open Distro for Elasticsearch. It naturally made it harder for Elastic to sell licenses to its proprietary features. It also added fuel to the fire in regards to the trademark infringement, since the AWS service now offered a partially different technology using the “Elasticsearch” trademark. Not only had Elastic not allowed for the trademark to be used in the first place, but now it wasn’t even the Elastic product being used anymore. Tensions were high, and in September 2019, Elastic filed a lawsuit against AWS for trademark infringement and false advertising.

License Changes

In January 2021, Elastic had enough and announced that they would change their licenses. Specifically, the Apache 2.0-licensed part of Elasticsearch and Kibana would be changed to the SSPL license – short for Server Side Public License. This is a license that was made by MongoDB Inc. for pretty much the same reasons; ensuring that cloud providers can’t use the software to offer a managed service without contributing back to the project. This was also a pain point for Elastic besides the trademark infringement; they felt that AWS was benefitting too much from Elastic’s software and contributing too little. Over the years, a number of tech companies have done pretty much the exact same thing as Elastic, some of which you can see here. Besides MongoDB, another notable example is Redis changing their licensing in March 2024.

The license change essentially meant that cloud providers would not be able to offer Elasticsearch and Kibana as a managed service without collaborating with Elastic. The source code would still be publicly accessible, but the software would no longer be considered open source by the Open Source Initiative. That’s why Elastic now brands the software as “open” and “free” instead of “open source.”

These days Elasticsearch and Kibana are generally provided under the Elastic License. Without going into detail, you can pretty much do everything except providing the products as a managed service and circumventing license key functionality.

The core of Elasticsearch and Kibana are still free to use with the source code being available, while certain features require a license. That’s pretty much the same as before, so the license change didn’t really affect users; you can still just download Elasticsearch and run it without needing to pay for a license. The license change was therefore very targeted at cloud providers and AWS in particular.

Introducing OpenSearch

Just a week after Elastic announced the license changes, AWS announced that they would develop an open version of both Elasticsearch and Kibana.

In April 2021 – i.e. a few months after the license change – AWS announced the successor to Open Distro for Elasticsearch: OpenSearch. This came as a direct response to the license change because it meant that AWS would no longer be allowed to offer Elasticsearch version 7.11 and newer as a managed service. So while Open Distro for Elasticsearch was a way to offer certain commercial features for free, the very core of Elasticsearch was still needed to make these work. Versions before 7.11 were not affected by this change, but being limited to old versions would not be sustainable. That’s why AWS created OpenSearch, which is a fork of Elasticsearch version 7.10, being the last Apache 2.0 licensed release of Elasticsearch. In case you are not familiar, a fork is essentially a copy of the software. AWS has then been developing this fork since without any affiliation with the original Elasticsearch and Kibana. Kibana was forked too, which is developed under the name OpenSearch Dashboards.

Eventually, this resulted in a settlement in the lawsuit between the two companies, as AWS would no longer be using the Elasticsearch trademark. That’s because they renamed their managed service to Amazon OpenSearch Service. The details of the settlement are not publicly known, but Elastic did make a claim for AWS’ profits from using the Elasticsearch trademark, so I can only imagine that they were compensated somehow.

Elastic took aim at AWS for using its software without contributing much back, while AWS took the high road and neglected Elastic’s open source spirit due to monetizing some features. I try to be fairly objective in my assessment, but I also think it’s important to remember that both companies have commercial interests in all of this, despite what they publicly say. Elastic probably didn’t want to miss out on the popularity of Amazon’s Elasticsearch Service and wanted to benefit from it somehow. AWS has not been a saint in this either, as they wanted control over the software they were basing a managed service on. Other companies besides AWS are contributing to OpenSearch, some perhaps out of their own commercial interests. That being said, AWS has retained control over OpenSearch and even has a trademark on the name. Elastic wanted a return on their investment for developing Elasticsearch and Kibana, while AWS wanted more control. At least that’s how I see it in general. Both parties had commercial interests in this and each of them have fair points. The truth is probably somewhere in the middle.

Comparing Elasticsearch & OpenSearch

Alright, so let’s compare Elasticsearch and OpenSearch. Although they are developed separately, they still share a lot of functionality. The core APIs are identical, so things such as field mappings, indexing documents, searching, aggregations, etc. are the same. The two parties are also inspired by each other regarding some of the features that have since been added, although I’m sure neither of them will admit that.

That’s not to say that there are no differences, though. The main differences are related to what I call “ecosystem solutions” or “vertical solutions.” For instance, Elastic provides very integrated solutions for security analysis, observability, enterprise search, AI, etc. These solutions tightly integrate various parts of the Elastic Stack, such as Elasticsearch and Kibana, and are more commercial in nature – at least for the advanced features. The result is a set of solutions that are powerful and easy to use out of the box. Generally speaking, they are provided as-is based on how Elastic envisions observability should be, for example.

OpenSearch, on the other hand, has a more open source approach that relies more on plugins and extensibility. Plugins for many things such as alerting, observability, and security, are bundled with OpenSearch, though, so this is not necessarily a bad thing. I personally find that the solutions provided by Elastic are a bit more user friendly and integrated, but it does come at the cost of requiring a license for some things.

I won’t get into a full feature comparison because then I should be writing a book instead. The most important thing to know is that the core is very much the same between Elasticsearch and OpenSearch, and the differences are most apparent in regards to the vertical solutions such as observability.

Choosing Between Elasticsearch and OpenSearch

Okay, so which one should you choose then? As you can probably imagine based on all of this information, the answer can be complicated. The good news, however, is that if you are just starting out, it doesn’t matter that much. That’s because the core APIs for indices, documents, searching, etc. are identical – for the most part at least. So if you are just getting started learning the technology, you can start learning Elasticsearch and then later switch to OpenSearch without many problems – and vice versa of course. So you can basically just choose the one you think you will end up using or just flip a coin. Once you know the core of either of the technologies, you should be able to make a qualified decision of which one to use long term – and without causing yourself too many headaches in the process.

The choice is more important if you A) need to use the vertical solutions that I just talked about, or B) if you need to decide which technology to deploy in production as a long term solution. In these cases the choice depends on a few factors, some of which are as follows.

Hosting

How do you want to host Elasticsearch or OpenSearch? If you want to host Elasticsearch yourself, you will need a license to use some features, which is not the case for OpenSearch. Either one can be hosted on any VPS or dedicated server, but keep in mind that managing a reliable and fault tolerant cluster is not trivial. If you want to use a managed service, you should look into which options are available, including their pricing and features. If you already use AWS, for instance, it might make sense to look into OpenSearch because AWS has a managed service for that. This service integrates well with other AWS services such as Kinesis, which may make things easier for you. The same is the case for the cloud providers that offer managed services for Elasticsearch.

Elasticsearch is well supported by the major cloud providers with the exception of AWS, for the reasons we discussed. Note that Elastic Cloud is built on top of three cloud providers; Google Cloud, Microsoft Azure, and AWS. When creating a deployment, you can choose which cloud provider’s hardware to deploy the cluster to, including the region. For example, if you choose AWS, they are responsible for the underlying hardware – i.e. EC2 instances – but it is Elastic that manages the software that runs on them. So while this does enable you to deploy Elasticsearch on AWS hardware, everything is managed within Elastic Cloud, and you also don’t get the integration with AWS services that you would with OpenSearch. Also worth noting is that when you deploy Elasticsearch on Elastic Cloud, you automatically get access to the commercial features that otherwise require a paid license. That’s because the license is “baked in” you could say. The same is the case for cloud providers that offer managed Elasticsearch services in a partnership with Elastic, such as Google Cloud and Microsoft Azure.

There are more providers that offer OpenSearch as a managed service, so that does offer some flexibility. Some of them offer very competitive pricing compared to the big cloud providers, especially for small clusters. Most providers offer a free way to try out their service.

Vertical Solutions

If you want to use vertical solutions such as observability or security analysis, then you really just need to choose if you prefer the Elastic or AWS way of doing it. If you just need to index documents and run search queries, aggregations, and the like, then either one will work just fine for you. And remember, there is nothing stopping you from using Elastic Cloud for observability and OpenSearch for search functionality, for instance.

Licensing

Finally, is it a problem that not all Elastic features can be used for free – at least without using a managed service? It probably depends who you ask and whether or not you intend to host a production cluster yourself.