An overview of Artificial Intelligence in recommending user products in the ecommerce industry

While the conventional method of shopping exists, the e-commerce industry has taken the world by storm. When everything from clothes to medicines is just a click away, artificial intelligence has become the driving force in the development of the eCommerce industry. The success of every eCommerce industry lies in its recommender system. Recommender systems are literally the systems that recommend products based on the users’ preferences. These recommendations are made by comparing and understanding their search histories and behavior. Artificial intelligence techniques are used to provide more accurate recommendations. A wide range of algorithms are prescribed for using recommender systems but is often difficult to finalize the more suitable algorithm. This paper puts forward the basics of recommender systems and how the implementation of artificial intelligence plays a crucial role in its improvement.

The eCommerce industry has become one of the leading industries of this era. As more and more people began preferring online mediums of shopping, the competition within this industry has crossed levels. Providing services customized to one’s likes and dislikes is quite challenging. Mainly because people’s preference is different from one other. Such customizations solve the problem of storing piles of data which in turn can provide a quick and efficient user experience.

The very first recognition of an Artificial intelligence-driven era was when IBM’s dark blue computer won against the human world chess champion. [1] From there on, AI has gathered success all along. It was twenty years ago when the first recommender system was developed by employing new techniques and theories taken from other AI mediums for understanding the users’ preferences. Artificial intelligence which has already shown its positive impact in industries such as healthcare, marketing, and finance, is all set to take over eCommerce.

A variety of techniques have been employed in the recommender system so as to enhance the easy accessibility and comfort of the user. Since AI is one of the most advanced technology this generation has come across, its recommendation skills can not be compared to conventional methods. For this reason alone, the user-product relationship can easily be understood when a user engages with eCommerce. In recent times, eCommerce has gained importance due to its added convenience of sitting in the comforts of our homes.  According to a market research firm, more than 80% of the jobs will be replaced by AI. [2]

Another important fact about recommender systems is that they help the user find the most desirable item from the huge data stored in one’s database. Initial applications were about information spaces like video recommendation, this proved to develop applications where a large group of users can identify the items they are most interested in. [3]

Literature Reviews

Robert Driskill [4] in the paper discusses the pros and cons of AI-based recommender systems. He says that recommender systems have the potential to become ubiquitous in eCommerce. But also it underlies many challenges. Hybrid data, accurate predictability, and content are some areas that require much thought.

Xia Song [5] The paper talks about progress in AI-led eCommerce. With its rapid development, deep learning platform, biometrics, NLP analysis, and other AI technologies will promote eCommerce industries. 

Sarker [6] The paper talks about various methods of machine learning to provide solutions for real-world problems. The machine learning model depends on the data provided and the performance. Several applications where machine learning is used in the real world are also discussed.

Badrul [7] The paper discussed a new approach in improving the scalability of recommender systems using clustering techniques. This could provide comparable prediction property as that of CF and also improve the online performance.

N CHA [8] The paper puts forth the users’ preferences by their types namely, category preference and attribute preference. The products are also classified based on their properties such as tangibility.

Advantages

Since the inception of eCommerce, recommender systems have progressed upwards. Ecommerce and recommender systems go hand-in-hand. These recommender systems are commonly found on large websites such as Amazon, Flipcart, Netflix, etc. Now, what makes these recommender systems so much more beneficial is the following unobvious list.

1. Increase in revenue and sales [9]: Companies using recommender systems profoundly have an increase in sales by a huge percentage. Since it can be used to solve different business goals, its popularity has risen. Consider the case of Amazon, there was an increase in 29% annual sales when recommender systems were successfully implemented.

2. Amazon used machine learning algorithms to process the data. A neural network was created to predict the preferences of the user. This in turn increased the sales and made a huge profit for the company. Another aspect is the review corner. Honest reviews of other users create trust in the product and the website. They suggest the firm to their friends and relatives which increases sales.

3. Growth in user satisfaction: Creating techniques that can increase the satisfaction of the user is one of the major goals of an eCommerce firm. When the user sees the feed personalized to their preferences, they feel more connected to the service. Be it in electronics, clothing, or books. So as they get comfortable with the website, they tend to stay longer and buy more.

4. Increase in turnover: As the recommendation system is more precise and accurate, the users begin to rely on that particular eCommerce more. The recommendation system analysis the user’s behavior and draw a pattern for future engagements. Its capacity is not limited to a particular user. It can take into consideration the connection between several users.

5. Personalization: When a user plans to buy something, they seek opinions from a lot of people. This is because they would be able to suggest the best product in terms of their taste. A recommendation system works on the same principle. It customizes products to the individual’s liking.

6. New discoveries: Sometimes the recommendations provided are exactly what the user would prefer. It usually contains other products similar to the user’s taste. This makes the website more reliable and provides user satisfaction. In such a scenario, users are bound to visit the site again in the future.[10]

Methodology

With increasing developments in technology, the users of the internet are presented with a large number of choices. This can be tedious and time-consuming. Recommender systems were initially designed to assist different users when presented with so many options. They use various sources of information to predict the user’s most likely choice. Recommender systems were first used in eCommerce to solve the information overload problem caused by Web 2.0  and were further used in other e-services[11].

The core element of the recommendation system is

f : U * I –> D

This is a function used to define the utility of a specific item ‘I’ to the user ‘U’. ‘D’ is the final recommendation list containing a set of items ranked according to the utility of all items the user has not used. The utility is presented in terms of ratings. The utility function is maximized so that the recommendation system can predict the user’s choice.

Predicting the utility of items for a specific user varies according to the recommendation algorithm that was selected. In all, the recommendation systems can be divided into content-based, collaborative filtering(CF)-based and knowledge-based [12].

ALS (Alternating Least Squares) is an implicit algorithm for the recommendation. In this algorithm, for every iteration, it tries to arrive closer to a factorized representation of the original data. ALS simply means fitting some lines to the data and calculating the sum of squared distances from all points to the lines and get an optimal fit.

Recommendation Steps:

Making recommendations is by calculating the recommendation score. as follow:

Recommendation score = Ui.VT  [13]

Where Ui = user vector and VT = Item vector.

After calculating the recommendation score, the scores were sorted in descending order to provide the top 10 recommendations.

This method can also be used in calculating the similarities among different products. The formula followed is as follows:

Similarity Score = V.ViT [14]

Where Vi = Item vector and ViT = transpose of Vi

Functionality

The function of recommender systems is based on understanding the user’s requirement. Three main types of relationships are formed[15]:

•  User Product Relationship: This occurs when the user has a preference for a specific product. For example, a guitarist would prefer websites related to guitars and strings. So the recommendation system makes it a point to include them.

• Product-Product Relationship: This is when items are similar in nature, either by appearance or description. For Example, books, and movies of the same genre.

• User-User Relationship: When some users have a similar taste with respect to a particular item. For example, mutual friends.

Recommender systems utilize the following data:

•         User Behavior Data

•         User Demographic Data

•         Product Attribute Data

Different AI techniques have enhanced recommender systems. The following are some:

1. Deep neural networks in recommender systems: This involves more classifying the data than ranking. This has achieved great success in NLP and speech recognition. As more data becomes available, the need for organization and integration increased the demand for deep networks in recommender systems. They are further divided into:

•         Multilayer perceptron-based recommender systems

•         Autoencoder-based recommender systems

•         Convolutional neural network-based recommender systems

•         Recurrent neural network-based recommender systems

•         Generative adversarial network-based recommender systems

•         Graph neural network-based recommender systems

2. Transfer learning in recommender systems: In recommender systems, transfer learning extends recommendation requests from a single domain to a multiple domains. This would help in exploring similar fields of interest and avoid data-sparsity and cold-start problems. This has led to the development of cross-domain recommender systems(CDRS). CDRS is further classified into:

•         CDRS with side information

•         CDRS with non-overlapping entities

•         CDRS with partially or fully overlapping entities

3. Active learning in recommender systems: This is introduced to assist the recommender systems to select the most representative item and deliver it to the user to rate it. This ensures greater efficiency and accuracy of recommender systems.

Implementation

With the usage of algorithms and data, recommendation engines filter and recommend products to the user. Algorithms are used to make the predictions more accurate. Recommendation systems process data in four steps: collecting, storing, analyzing, filtering[16].

• Collecting data: Recommendation systems require data. The collection of data is the first stage. Data are classified into implicit and explicit data. Explicit data are the ones like reviews and ratings of other users. Implicit data are the search histories, search logs, clicks, etc. These kinds of data are collected from any users when they encounter an eCommerce firm. 

• Storing data: More data should be available for the recommendation system to be accurate. Storing data can be either using a NoSQL database or an SQL database. They capture the data input and a scalable database decreases the number of tasks to a minimum and focuses on the recommendation.

• Analyzing the data: Analyzing the data stored is quite important in a recommendation system. Users require results immediately; real-time systems are used in such a case. They are capable of processing the data as and when it was created. Near-real time systems are used in the case of the same browsing session. It gathers the data quickly and refreshes the analytics for a few seconds or minutes. Whereas batch analysis is a more convenient method in sending emails at a later time.

• Filtering data: The final stage is filtering data to provide the best recommendations to the user.

i. Content-based filtering focuses on a specific shopper. It makes use of the content of an item. The algorithm follows visited pages, time taken on a category, etc. The software is created on the basis of user likes or dislikes of the product. Initially, the properties of different items are extracted from documents. They profile a user’s preference from their consumption records. As and when the user profile is sorted, the recommendation system compares the product and the user’s profile to construct a recommendation list. Finally, the relevant products are forwarded and those which the user could dislike are filtered out. 

ii. Knowledge-based filtering is based on the available information about a user’s needs and item functions. These type of systems retain the knowledge which was extracted from the user’s histories. It contains previous problems and their solutions. When the system encounters a new problem, the knowledge base is referenced.

iii. Collaborative filtering makes predictions of the user’s preference and makes product attributes. They are based on the user’s historical records. They take into consideration the ratings. The basic principle being that users who share similar interests will buy similar items. So collaborative filtering depends on the users who have similar preferences to the given user. They are further classified into memory-based CF and model-based CF. 

Approaches to content-based recommender systems:

As discussed, content-based systems work well with descriptive data when provided beforehand.

Approach 1: Using rated contents to recommend

In this approach, recommendations are based on already rated products based on the user’s preference, then a rating is predicted for a similar product.

This is helpful when a product has no user reviews.

Approach 2: Through the description of the content

This approach looks into the description of the product before making a recommendation. The description could be the title, summary, tag lines, etc of the product. Since the format is in string format, it has to be converted into a numeric format for easy calculation.

The implementation step also include: [17]

• Data preparation and cleaning

This includes data extraction and removing unnecessary data.

• Creation of space matrix

A space matrix consists of the user and MCAT data.

• Calculation of user vectors and MCAT vectors

They are calculated using the same formula discussed earlier.

• Recommendation of MCAT to users

When the user and MCAT vectors are created, the recommendation scores are calculated and are arranged in descending order.

• Speeding the process and saving the model file

Result and Discussion

Recommender systems with artificial intelligence have been introduced to facilitate the high popularity of recommender systems. By indulging AI, the user can get accurate results when scrolling through websites. This is because they use machine learning algorithms that can store and analyze the user’s data to provide a reliable product. In a competitive world, product recommendation systems are a go-to for every retailer. This paper discussed the basic concept of a recommender system and how implementing artificial intelligence can enhance an eCommerce industry.  

Conclusion

The recommender system is one of the leading technologies in the modern era. They extract information from the database so as to create additional value for the business. These systems help the user in choosing the product they like when the user visits an eCommerce firm. While the initial goal of recommender systems was to reduce the information overload for internet users and make information retrieval more efficient, it has now become a crucial strategic tool for firms in e-business. Moreover, the recommendation box on the content page can be very persuasive because it can also recommend similar items. In all, there is nothing holding back from implementing a successful recommender system.

Every user-item relation in the recommender system is necessary for profiling user preferences. The greater number of reviews received from the user increases the betterment of the recommendation system. Artificial intelligence plays a huge role in the success of every eCommerce industry. Since the competition in the same is quite high, fast and precise are the options every user looks for. From the user’s point of view, it creates the experience and creates engagement. From the retailer’s point of view, it is an attractive bet. It generates more revenue. It is better to have a basic recommender system for a small set of users and invest in more powerful techniques once the user base grows. More importantly, business goals will dictate the type of recommender system you should focus on at first: whether it is generating more engagement for already active users or pushing those infrequent customers to become more active. Besides that, analyzing information that has gathered is the key.

Below is the compression study between two major methods in the recommendation system.

Method	Application	Advantages	Limitation
Collaborative Filtering	User-based – To predict the product that the user might like based on ratings given to the product by other users who have a comparative taste with that of the target user. Item-based – This type of recommendation approach pools similar products based on the product that users have already liked.	No product knowledge required as we pool the user with similar taste.	This approach cannot work for new items.
Content-based Filtering	This method predicts with user past webpage visit, time taken for the category, previous purchase, and feedback history	This approach produces more accurate suggestion and can recommend even niche products with fewer data samples	This approach requires a lot of domain knowledge and has limited ability to suggest precise products if the content doesn’t have much information.

Author: Arun Poochelvan

References

1. Zhang, Q., Lu, J. & Jin, Y.“Artificial intelligence in recommender systems.” Complex Intell. Syst. 7, 439–457 (2021).W(https://link.springer.com/article/10.1007/s40747-020-00212-w)
2. Xia Song, “The application of artificial intelligence in electronic commerce”, Journal of physics, conference section, 2019(https://iopscience.iop.org/article/10.1088/1742-6596/1302/3/032030/meta)
3. Robert Driskill and John Riedl, “Recommender systems for ecommerce: challenges and opportunities”, AAAI technical report (https://link.springer.com/chapter/10.1007/3-540-45675-9_33)
4. Azati team, “Recommendation Systems: Benefits and development process issues”, June 3, 2021.( https://w9.guide-college-journal.bar/244)
5. Maruti Techlabs, “5 advantages recommendation engines can offer to business”, October 5, 2017.( https://towardsdatascience.com/5-advantages-recommendation-engines-can-offer-to-businesses-10b663977673)
6. Lu J, Wu D, Mao M, Wang W, Zhang G (2015) Recommender system application developments: a survey.( https://www.sciencedirect.com/science/article/pii/S0167923615000627)
7. Rodrigo Schiavini, “AI Product Recommendation Engine”, Smarthint.( https://www.smarthint.co/en/ai-product-recommendation-engine/)
8. Badreesh Shetty, “An in-dept guide to how recommender system work”, Builtin, July 24, 2019.( https://builtin.com/data-science/recommender-systems)
9. Sarker, Iqbal H. “Machine Learning: Algorithms, Real-World Applications and Research Directions.” SN computer science vol. 2,3 (2021).( https://pubmed.ncbi.nlm.nih.gov/33778771/)
10. Badrul, George, Joseph, John, “Recommender Systems for large scale e-commerce: scalable formation using clustering” , Army HPC Research center.( http://glaros.dtc.umn.edu/gkhome/fetch/papers/clusterICCIT02.pdf)
11. N. CHA, H. CHO, S. LEE and J. HWANG, “Effect of AI Recommendation System on the Consumer Preference Structure in e-Commerce: Based on Two types of Preference,” 21st International Conference on Advanced Communication Technology (ICACT), 2019(https://ieeexplore.ieee.org/document/8701967)
12. Ashutosh Singh, “Recommendation system for ecommerce using collaborative filtering”, Medium, 22 July, 2019(https://medium.com/@ashutoshsingh93/recommendation-system-for-e-commerce-using-collaborative-filtering-fa04d6ab1fd8)
13. Ivens, Paulo, Donald, “The use of machine learning algorithms in recommender systems: A systematic review”( https://www.researchgate.net/publication/284219925_The_Use_of_Machine_Learning_Algorithms_in_Recommender_Systems_A_Systematic_Review)
14. Roger Chua, “A simple way to explain the recommendation engine in AI”, Voicetech podcast, 26 June 2019.( https://medium.com/voice-tech-podcast/a-simple-way-to-explain-the-recommendation-engine-in-ai-d1a609f59d97)
15. Countants, “How artificial intelligence is transforming the ecommerce industry”, 10 May 2019.( https://medium.com/@Countants/how-artificial-intelligence-is-transforming-the-e-commerce-industry-countants-scalable-custom-73ae06836d35)
16. Rodrigo Schiavini, “AI Product Recommendation Engine”, Smarthint.( https://www.smarthint.co/en/ai-product-recommendation-engine/)
17. Badreesh Shetty, “An in-dept guide to how recommender system work”, Builtin, July 24, 2019.( https://builtin.com/data-science/recommender-systems)