One of the most important features in an e-commerce retail solution is the capability to provide high-quality recommendations for buying other products/items. A system responsible for this feature is known as a Product Recommendations (PR) system. A good PR system can significantly increase the website profits by driving additional product sales for the existing customers and also by increasing the customer base. Ultimately, from the end-user point of view, the product recommendation is that feature that provides this type of information:
Product recommendations systems can be found everywhere in the modern e-commerce world; below some relevant numbers (source):
As a proof of the importance of having good product recommendation systems, a few years ago Netflix organized a challenge (the “Netflix prize”) where the goal was to produce a recommender system that performs better than its own algorithm, with a prize of 1 million dollars to win.
So let’s go together through the main concepts used in building recommendation systems, what is at the core of such systems, and why using our multi-model database – ArniaDB is the right technology choice to build a product recommendations solution.
There are three main models used to design and implement a product recommendation system:
The usual components we see in a product recommendation system are:
Depending on how often the data is updated, there are two types of PR systems:
Obviously, real-time systems provide the most accurate recommendations, and they should be implemented whenever possible. However, in practice there are constraints that limit the ability to always implement a real-time system due to:
Therefore, it might be a good idea to start with an offline system design, and when the system is complete, up and running to the targeted parameters, make the adjustments to transform it into a real-time system.
There are many reasons for choosing ArniaDB as your database component for a high-quality PR system:
As opposed to most commercial solutions, such as Oracle or MS SQL Server and others, ArniaDB comes with very affordable pricing plans. We also offer a free community edition.
Our team has extensive, rich experience in building database solutions, with a proven track of success. Most of our engineers have more than 10 years of experience in this area; we have also specialized consultants with vast knowledge in building solutions using back-end databases. Read more about our team here.
The assumption is that you have selected your e-commerce application solution (such as Magento) and what you need on top of it is a high-quality PR system. There are two main choices to select from:
In this case, the assumption is that you have already in place an e-commerce solution (Magento, Shopware etc.), having a PR system in place based on a content-based filtering method, and you need is to add a high-quality PR system based on a collaborative model. There are two main choices to select from:
The main differentiator between using ArniaDB and other databases is the multi-model capabilities built-in in the ArniaDB. Having both relational and graph apabilities in the same unified database engine provides important advantages toward design, implementation, maintenance, performance, and overall costs.
The current market solutions design for a hybrid model is usually the following:
But by using ArniaDB the system design is optimized for data processing, organized by specific functional responsibilities, more performant and easier to maintain. Another potential advantage is that the e-commerce relational database is not “polluted” with PR specific processing data – everything related to PR implementation/processing can be stored in a single database – ArniaDB (such as PR analytical data).
In all scenarios, our services cover all the solution implementation phases:
We will do a complete analysis of your specific requirements and will recommend the selection of methods and algorithms that will best fit your needs. For content-based filtering implementations we will analyze the fit of the most used models, such as:
SELECT c.CustomerId,
p.Model,
COUNT(p.Model) as FrequentBuy
FROM …
GROUP BY
p.Model,
c.CustomerId
ORDER BY FrequencyBuy DESCFor collaborative filtering, the analysis will target the fit for known models such as correlation-based, cosine-based, dimensionality reduction techniques, latent semantic methods, regression, clustering, neural networks, particle filtering and others.
MATCH
(customer:Customer {name: ‘Joe’})-[:BOUGHT]->(:Basket)<-[:IN]-(product:Product)
RETURN product, count(product)
ORDER BY count(product) DESC LIMIT 10
Based on the analysis results, we will propose the best solution, and we will build the solution architecture and the system design. This includes the design of the data storage (graph database & relational database, as applicable).
As a side note, a good solution must address the typical issues associated with PR systems:
For both models, in particular for the collaborative filtering model which requires a graph database back-end, we need to select, prepare, and load the data needed to run the algorithms (i.e. populate the dedicated graph and relational storages). This is achieved by implementing an ETL (Extract-Transform-Load) module/application.
The ETL solution will be executed at the scheduled periods or continuously, for real-time solutions. For example, it is very important to have an ETL batch process that will calculate in parallel the various similarities between the system entities in scope (such as users, items) when you scale to big data (millions of entities), in order to provide fast response times.
In this phase we will implement the product recommendations engine, using one or multiple (hybrid) model(s).
Our product team will ensure proper system functionality, periodic updates, solution improvements and evolution.
Not the least, as part of the solution, we will implement tools to support specific metrics and efficiency data analysis. The quality of the PR system must be constantly measured, in order to be able to act for improvements.
It is not just users and products/items that can be used in a recommendation system. A recommendation system is a more general domain than just what we see in e-commerce product recommendation solutions – it is any system that provides a recommendation, prediction, opinion, or user-configured list of items that assists the user in evaluating items.
What is important is that the same theory, models, techniques, and solutions used for e-commerce PR systems can be applied in other domains and scenarios, beyond the typical e-commerce online store, such as for:
If you are interested in a recommendation system that is not used for an e-commerce solution, don’t hesitate – we can absolutely do it together, so please contact us.
Making recommendations is mostly about taking past behavior and preferences and trying to predict a preference. But it is not an exact algorithm. A certain degree of unknown is always present. The recommended products can depend on factors which are not easily considered in a “standard” system, such as:
All these can influence the quality of the recommendations, which is measured by the sales volume increase. To build a more flexible PR system that can learn by itself over time and make connections that are not immediately spotted by the usual models/algorithms, more and more we see Machine Learning and Artificial Intelligence engines added to the PR systems.
We can put to work our rich experience in designing and building ML/AI solutions to create or enhance new or existing recommendations systems.
We have the right people and skills, a rich and very successful software development experience, the knowledge about e-commerce solutions, and the right technologies – ArniaDB in particular, to build performant and efficient recommendations solutions that will help your e-commerce business to sell more products and to increase the customer base.
Please contact us to analyze together your specific requirements and to propose the best solution for your business.