How an Expense Management Leader Optimized Their Elasticsearch Through a Massive Upgrade
ABOUT THE CLIENT:
SERVICES USED:
SERVICES USED:
The Challenge
Our collaboration with this client started in 2011, and it expanded to several different projects. We have worked on our client's existing applications and developed a white-label mobile app.
The Solution
Our expense management client had a very complex Elasticsearch system with over 10 different types of searches. This client prided itself on high-quality results and wanted to be assured that the new search capabilities would work like their past version. Because the search query is built differently between the versions, we developed a comparison tool to ensure the accuracy of the new search before the client launched the product to their users.
The Result
The client’s Elasticsearch is now hosted on AWS infrastructure, and their users will have the same experience or even faster when they use the search functionalities. And as an additional perk, our client can use the custom comparison tool for future Elasticsearch upgrades to ensure accuracy.
The Challenge
Our collaboration with this client started in 2011, and it expanded to several different projects. We have worked on our client's existing applications and developed a white-label mobile app.
Due to our long-term collaboration, it was great to work on their latest project, upgrading their Elasticsearch from version 2.3.4 to version 7.9. Because our client is driven to offer their users a seamless experience, they wanted to make sure that an upgrade to their Elasticsearch would not impact how their users interacted with their service.
Elasticsearch upgrades are essential for many reasons. Every new version of Elasticsearch offers users a chance to get more from their deployment and have new features, fixes, and enhancements. Plus, when it comes to clients working in the finance industry, data is everything — for payments, transactions, ATMs, branches, APIs, devices, and more. All that data can become a strategic asset if managed correctly, and utilizing Elasticsearch is a great way to do so.
It’s a common story we hear from clients: they set up an Elasticsearch cluster/server and don’t always make it a priority to keep it up-to-date with the latest version. After a while, they realize that they are several major releases behind the stable version, and there’s no straight path to upgrade.
Some companies put Elasticsearch upgrades on the back burner because there are higher priority items they feel they need to accomplish. No matter the reason, postponing an upgrade can pose many challenges for your future business ventures.
By holding off on updates, it is much more difficult and time-consuming to make the leap to the next version. It is much easier to update from, let’s say, version 6.0 to 7.9 than it is to update from version 2.3.4 to 7.9. Besides the difficulty of upgrades when making a large leap between versions, there are other challenges you might face when deprecating your Elasticsearch, including:
- Longer downtime during upgrades
- Endpoints with different input parameters
- Lucene syntax errors between versions
- Performance issues
However, even though there are obstacles to securing a newer version of Elasticsearch, most clients find an upgrade extremely worthwhile since each new version of Elasticsearch provides a much faster, easier to use, and more secure and resilient user experience.
Additionally, new versions come with many enhancements and new features - and in the case of Elasticsearch version 7.9, an endpoint for bulk deletion, which makes it easier to replace and clean up data. Overall, an upgrade to Elasticsearch version 7.9 will provide a world of difference compared to our client’s past 2.3.4 version, especially regarding speed and scalability.
Elasticsearch 7.9 also provides the client’s users with the following benefits:
- Performance Improvements
- New Features, like bulk deletion
- Improved bug & security fixes
- Faster searches
- More accurate results
The Solution
Fortunately, even when our clients are a few versions behind on Elasticsearch upgrades, Softjourn is capable to make smooth upgrades happen. When we plan a big upgrade, we always consider, and have plans for, our clients’:
- Security
- Inventory
- Testing needs
- Breaking changes
- Deprecation
- Prior & future monitoring
- Troubleshooting
- Mapping Issues
- Analyzing the health of our client’s existing system
Our expense management client had a very complex Elasticsearch system, with over 10 types of different searches. This client prides itself on high-quality results and wanted to be assured that the new search capabilities would work the same as their past version.
Because the search query is built differently between the versions, we developed a comparison tool to ensure the accuracy of the new search before the client launched the product to their users. We used the following process to perform and assess their upgrade and prove the accuracy of the search:
- We developed some Java microservices from scratch
- We created a custom comparison tool
- Before launching the upgrade, we evaluated the old and new search versions using the comparison tool
- We analyzed the percent of errors between searches in version 2.3.4 and version 7.9
- We investigated and provided insights on what could cause a difference in results.
- After changes, we re-compared the versions using the tool and found an acceptable percentage of the difference between the results (less than 2%)
The Benefits
Our comparison tool gave our client the confidence to launch the upgrade of their Elasticsearch. The latest searches from their users showed less than a 1% of difference between the old and new versions, our team managed to achieve a .02% difference for 500,000 search queries.
The client’s Elasticsearch is now hosted on AWS infrastructure, and their users will have the same experience, or even faster when they use the search functionalities. And as an additional perk, our client can use the custom comparison tool for future Elasticsearch upgrades to ensure accuracy.
By having access to a partner with knowledge of Elasticsearch upgrades and expertise in the finance industry, our clients will rest assured knowing that their end users will have the same (or even better!) experience after the upgrade.
Our team carefully researched and investigated exactly what was needed for the upgrade in order to make the process go as smoothly as possible.
Conclusion
With a newer version of Elasticsearch, our client will feel confident that its system will be resilient, speedy, scalable, and most importantly, secure. The performance improvements, new features, and security fixes gained through updating Elastic Search will have incredible value for them.
Plus, updating to new versions of Elasticsearch will be exceptionally easier and quicker for the client in the future, due to our customized comparison tool. This will save them time, money, and effort in the long run.
Our client was happy with their Elasticsearch upgrade, especially since they’ve had no errors reported by their users during or after the upgrade. Their users didn’t note any changes from the switch, besides a faster search speed, which is exactly what we hope for after an upgrade.
If you are interested in upgrades to help your business’ scalability, security, or ease of use, Softjourn is here to help.
Our experience working in the expense management & finance industries allows us to bring well-informed guidance to your project and our talented teams are eager to create a perfectly customized solution for your business. If you want to work with a highly skilled team to help you with AWS and Elastcisearch - contact us.