Digital news are arguably ones of the most consumed products by millions of people every day and the importance of a proper recommendation system has always been fundamental for news and press websites. KRED deals with a knowledge graph system, taking in account information from entity’s neighborhood, and refines the entity embedding with dynamic insights depending on the context of the news. This repository proposes two main improvements to the original model: a Multi-Head Attention module and an enrichment of side information encodings, such as category. The dataset used is MIND dataset, created on a huge amount of Microsoft News interactions between users and news.
KRED is a knowledge enhanced framework which enhance a document embedding with knowledge information for multiple news recommendation tasks. The framework mainly contains two part: representation enhancement part and multi-task training part.
Two extensions to this model have been implemented:
- in the context embedding layer, we enriched the entity emdedding with the news category, consisting of a first general category and a second more specific category
- in the information distillation layer, we replaced the self-attention module with a multi-head attention module
The backbone resources of this project are listed in this table, including papers and corresponding GitHub repos.
| Paper | Title | Implementation source |
|---|---|---|
| 1910.11494 | KRED: Knowledge-Aware Document Representation for News Recommendations | KRED |
| 2020.acl-main.331 | MIND: A Large-scale Dataset for News Recommendation | MIND |
- The code has been tested using the Google Colab environment, running under Python 3.8.10
- Multiple NVIDIA GPUs are mandatory to run the model in feasible time but, if you use smaller version of the dataset (MINDsmall), the number of GPUs needed can be reduced; we could run the model with Colab's GPU (Tesla T4 16GB) using MINDsmall
Once the repo is cloned, some python libraries are required to properly setup your (virtual) enviroment.
They can be installed via pip:
pip install -r requirements.txtor via conda:
conda create --name <env_name> --file requirements.txtThe dataset explored by KRED is Microsoft News Dataset (MIND), a large-scale dataset collected by anonymized behavior logs of the Microsoft News website. Constructed from the user click logs, MIND contains 1 million users and more than 160k English news articles; in addition, Microsoft has released a smaller version of the original dataset, by randomly sampling 50k users and their behavior logs (MINDsmall).
Each news consists of:
- News ID
- Category and Sub-Category
- Title and a brief Abstract
- URL of the news
- List of entities in the Title
- List of entities in the Abstract
Each entity is characterized by:
- Label, which the entity is associated to
- Type of entity, such as person, object ecc.
- WikidataId of the label
- Confidence of label
- Position of first char in the text
- Entity itself
Before running the code, you should check config.yaml and setup all the parameters correctly.
You can perfom the training and testing phases using kred_example.ipynb. If you want to use Colab, download kred_example.ipynb, then upload it on Colab, change runtime to use GPU and follow the instructions in the notebook.
| Author | GitHub | |
|---|---|---|
| Lorenzo Bergadano | s304415@studenti.polito.it | lolloberga |
| Stefano Gioda | s294781@studenti.polito.it | Giost |
| Paolo Rizzo | paolo.rizzo@studenti.polito.it | polrizzo |