6th RecSys:NL Meetup

The 6th RecSysNL meetup will be on Wednesday September 28, 2016. The meetup will be hosted by ORTEC. The address is: Condensatorweg 54, 1014 AX Amsterdam. The meetup starts at 17:30.

Please note that you need to sign up for a free ticket to participate. Tickets are available at our Eventbrite page.

There’s an interesting lineup of great talks by TelefonicaCLEF-NewsREEL/DAI-Labor of TU Berlin, and Xing.

“Deep learning in recommender systems” by Alexandros Karatzoglou

Deep Learning (i.e. the return of Neural Networks part deux) is one of the most active and interesting areas in Machine Learning at the moment. New Deep Learning methods have shown to perform in several tasks in Image Processing, Natural Language Processing and Signal Processing. The emergence of these new Machine Learning methods creates new opportunities in the area of Recommender Systems, while at the moment there is relatively little work in the intersection of Deep Learning and Recommender Systems I will try to give an overview of the existing work, and also try to make some educated guesses about the future of Recommender Systems in light of the advancements in Deep Learning.

Speaker Bio:

Senior Research Scientist at Telefonica Research in sunny Barcelona. I focus on several aspects of Machine Learning and Recommendation like Deep Learning, Learning to Rank and Context-awareness. Recipient of a two-year Marie Curie IEF fellowship award to undertake research on Context-aware Recommendation at Telefonica Research. My research in the area has been awarded with three best-paper awards at ECML/PKDD 2013, RecSys 2012 and ECML 2008.
Apart from mentoring PhD students at Telefonica Research, I am currently teaching courses on “Deep Learning” and “Machine Learning for Recommender Systems” at the UPF Graduate School of Economics Masters Course in Data Science in Barcelona and at the GSE Data Science Summer School.
I’m also the author of kernlab, a fairly popular Machine Learning package for R, and of CoFiRank, a collection of algorithms in C++ for Collaborative Ranking.
I received my PhD in Machine Learning from the Vienna University of Technology, while also being a frequent visitor at the Statistical Machine Learning group at NICTA in Canberra, Australia.
In my spare time, I enjoy kite-surfing, snowboarding, riding my bicycle, or learning to dance lindy hop.

“NewsREEL challenge: online evaluation of recommender systems” by Andreas Lommatzsch

Traditionally, the evaluation of recommender algorithms focuses on the offline evaluation using static, set-based datasets. For many web-based application scenarios the “static” evaluation does fit the characteristics of the scenario.
The News Recommendation Lab (NewsREEL) challenge wants to encourage researchers to focus on the more realistic online evaluation of recommender algorithms. The NewsREEL challenge allows researchers to evaluate algorithms for recommending online news articles. The algorithms are analyzed both online and offline taking into account the recommendation precision as well as technical aspects.
The talk explains the NewsREEL challenge in detail: We discuss the addressed challenges, review the implemented approaches, and explain our experiences gained organizing the challenge.

Speaker Bio:

Dr. A. Lommatzsch received his PhD degree in Computer Science from the Berlin Institute of Technology. His research focuses on distributed knowledge management and information retrieval and systems. Since 2009 he has been a senior researcher and project coordinator in the domain of Recommender Systems and Machine Learning. His primary interests lie in the areas of recommendations based on data-streams and context-aware recommender algorithms.

“Recommendation challenges on Xing” by Daniel Kohlsdorf

XING is a social network for business. People use XING, for example, to find a
job and recruiters use XING to find the right candidate for a job. At the moment,
XING has more than 15 Million users and around 1 Million job postings on the
Among other things, Xing’s data science team works on job recommendations.
In this talk we will introduce the job recommendation problem as well
as Xing’s ongoing work on selected components of our recommender system.
Topics will include content based recommendations with latent spaces,
outlier filtering with ensemble methods. Furthermore, we
will describe the recommender systems challenge 2016 and how
our team created a semi-synthetic dataset released to the public.

Speaker Bio:

Daniel is a Data Scientist at XING with a focus on data mining projects
and recommender systems. His research interests include automated behavior
analysis of humans and dolphins, contextual computing and data mining
with the application to recommender systems. At XING, he is developing
large-scale recommender systems that serve millions of users and handle
hundreds of requests per second. He received his PhD
in computer science from Georgia Tech with a specialization in machine
learning and computational perception and a masters degree in informatics
from University Bremen. He is an organizer of the RecSys Challenge at
ACM RecSys 2016 on job recommendation.


5th RecSys:NL Meetup – Wrapup

The 5th RecSysNL meetup was held on November 24th in Elsevier, Amsterdam - a great venue with drinks and nice food. Thanks to Craig Scott and Elsevier.

There was three interesting talks: First, Marco Rosseti talked about their recommender systems in Mendeley. He introduced Mendeley Suggest, a new research advisor that recommends research articles to read. It uses different data sources and algorithms to provide recommendations that satisfy a broad range of researchers’ information needs. The slides of his talk is available here.

Then, Rolf Drög and Mikel Porras Hoogland gave an overview of their social music platform called Kollekt.fm. They are working on a music recommender systems for Kollekt.fm, together with TU Delft within the CrowdRec project.  Finally, Artem Grotov talked about self-learning search engines. He proposed an approach in which the search engine learn how to combine numerous search criteria by observing how users interact with the search engine.

This slideshow requires JavaScript.


5th RecSys:NL Meetup

The 5th RecSysNL meetup will be on Tuesday November 24, 2015. The meetup will be hosted by Elsevier. The address is: Radarweg 29, 1043 NX Amsterdam. The meetup starts at 17:30.

Please note that you need to sign up for a free ticket to participate. Tickets are available at our Eventbrite page.

There’s an awesome lineup of three talks by Mendeley, Kollekt.fm, and ILPS lab of UvA.

Talk 1: “Research recommendations at Mendeley” by Marco Rossetti

Staying up to date for a researcher is becoming more and more difficult, as the number of conferences and journals increase and the subjects are constantly evolving and transforming. In Mendeley we are researching and developing recommender systems that help researchers to discover research and researchers that are relevant to them. In this talk Marco is going to present Mendeley Suggest, a new research advisor that recommends research articles to read. Mendeley Suggest exploits different data sources and algorithms to provide recommendations that satisfy a broad range of researchers’ information needs.

Speaker Bio:

Marco Rossetti is a Data Scientist at Mendeley and he is currently focused on researching and developing recommender systems for researchers. Marco earned his PhD in Computer Science in 2015 at University of Milano-Bicocca, working on recommender systems.

Talk 2: “Brands using music to connect to their customers” by Rolf Droge and Mikel Porras Hoogland from Kollekt.fm

Kollekt.fm a is social music platform; we have created a community of music lovers that curate high quality playlists for others to enjoy.
Kollekt.fm is currently exploring ways to connect these high profile music lovers to brands, so that they can provide music for their stores. Since we are living in this streaming era and music is so easily accessible, we see a massive opportunity, to not only use this music for the in store experience but bring it to the online world. This way brands can connect to their customers on a whole different level than they are doing at the moment. But to get the best music – in the most efficient way – Kollekt.fm is teaming up with the TU-Delft in the CrowdRec project to create a music recommendation system.

Speaker Bio:

Mikel and Rolf are two of the four co-founders of kollekt.fm. Both graduated from the TU-Delft; Mikel has a masters degree in Interaction Design and Rolf has a bachelor in Mechanical Engineering. They’ve always been huge music enthusiasts; Mikel as a DJ is producing music whereas Rolf has been in the event promoter space. 

Talk 3: “Self-learning search engines.” by Artem Grotov from University of Amsterdam

Search engines like Google, Yahoo and Bing provide users with high quality search results. They do so by taking into account thousands of factors that affect the order of the results shown on the search engine result page. These factors include textual query-website similarity, popularity of the website, the time since the website was last updated, click through rates and many others. It comes at a great cost – the search engine engineers have to create huge annotated datasets in order to learn how to combine these factors. Smaller companies cannot afford to go through this process, this is why we, as users often have poor search experience outside of large commercial search engines. For example the quality of search inside university websites, corporate websites and online stores is often very poor. In this talk Artem will discuss a potential solution to this problem. This solution is to have the search engine learn how to combine numerous search criteria by observing how users interact with the search engine. This way it is possible to provide excellent user experience without the cost of annotating large amounts of data. Additionally the search engine adapts to real users and not to the people who annotated the data and does so in real time.

Speaker Bio:

Artem Grotov is a PhD candidate at University of Amsterdam. His interests are machine learning, online learning and counterfactual evaluation.

4th RecSys:NL Meetup

The 4th RecSys NL meetup will be on Tuesday May 19th at TNO Leiden. The meetup starts at 17:30 followed by drinks and pizza. The address is: Gaubiusgebouw, Zernikedreef 9, 2333 CK Leiden.

Please note that you need to sign up for a free ticket to participate. Tickets are available at our Eventbrite page.

There’s an awesome lineup of four talks:

Talk 1: “Recommendation technology for media services.” by Martin Prins from TNO

The media industry is under pressure. There’s lot’s of competition, causing users to move to other services or to use services less frequently. This is a major concern for many media companies, as they typically rely on advertising revenue. TNO has recently been involved in research projects with several Dutch media companies, in which recommendation technology plays an important role.
In this talk Martin will present several case studies and shed insights into the role of recommendations for media services, the technical challenges that were faced, and findings of the research projects conducted by TNO.

Speaker Bio:

Martin Prins is a Technical Consultant at TNO within the Media and Network Services group. With an R&D background in multimedia, especially streaming media and broadcast TV, Martin’s current focus is helping broadcasting, news and media companies with their digital media innovations, in which personalization and recommendations play an ever more important role.
Martin holds a Master Degree in Telematics from Twente University.

Talk 2: “Recommend Now, Not in the Past: Leveraging Contextual User Profiles for Destination Recommendations” by Melanie Mueller & Lucas Bernardi from Booking.com

At Booking.com, we recommend destinations to travelers who are not yet sure where to go. Typical recommender systems rely on past user feedback to recommend items to users. This can be problematic in our real-world application, where user interactions are infrequent (sparse data), and where many users come in for the first time or change interest over time (continuous cold start problem). Here, we propose to use the current user situational context, instead of past user interactions, to inform recommendations. In an A/B test on Booking.com users, contextual recommendations increased user engagement by 20%.

Speakers Bio:

Melanie and Lucas are data scientists at Booking.com, where they enjoy transforming terabytes of data into recommendations for fun holidays.

Talk 3: “Large-Scale Real-Time Product Recommendation At Criteo” by Romain Lerallut & Olivier Koch from Criteo

Behavioral retargeting consists of displaying online advertisements that are personalized according to each user’s browsing history. As it is performed for each personalized banner, billions of times a day, the selection of the products to display in the banner needs to be fast and accurate.
At Criteo, we built a recommender system which is able to choose a dozen of relevant products from over two billion products in a few milliseconds. In this talk, we will expose the problems we faced whilst building this system and how we solved them thanks to a mix of online and offline computations.

Speakers Bio:

Romain Lerallut is a Senior Engineering Manager at Criteo in the Engine team, in charge of applying large-scale machine learning algorithms to actual problems such as product recommendation or graphical layout optimization. Romain has been working on building up the Engine since 2011, first as devlead and then as manager. He has an engineering degree from “Ecole des Ponts-Paristech” and a PhD in Computer Science from “Ecoles des Mines-Paristech”.

Olivier Koch is an Engineering Program Manager at Criteo, with a specific focus on prediction algorithms and recommender systems. Prior to joining Criteo, Olivier worked as a software engineer and team manager in the fields of image processing at General Electric Healthcare and Thales Optronics. Olivier holds an engineering degree from ENSTA-Paristech and a PhD degree in Computer Science from Massachusetts Institute of Technology.

Talk 4: “Musical similarity and recommendations for DJ apps” by Victor Bergen Henegouwen from Elephantcandy

Music is a difficult matter to analyse; unlike text, there is no unambiguous language and notation. Transcription, notation or annotation of music typically describes a very limited set of features and is highly decoupled from the actual audio.
In Electronic Dance Music (EDM), this problem is even more pronounced; notation rarely exists and common structural elements like choruses and couplets are absent. Instead, the most distinct features of a dance track are the ‘sound’ and (very) subtle variations in rhythm. Together with the ILLC department of the UvA, Elephantcandy has been working on a system to extract features from the audio that are relevant in EDM, make them comparable and recommend parts of songs that can be mixed well together.

Speaker Bio:

Niels Bogaards is co-founder and CTO of Elephantcandy, an Amsterdam-based app development company focussed exclusively on mobile audio. Previously he worked at IRCAM in Paris as researcher/developer in the Analysis-Synthesis Team.

3rd RecSys:NL Meetup – Wrapup

The 3rd RecSys meetup was a great success with around 50 participants from all over the Netherlands and from both academia and industry. It was held on December 8th at SDL, Amsterdam - a great venue with drinks and nice foods. Thanks to Julian Ortega and SDL.

There was an interesting mix of four talks:

Jakub Zavrel (from TextKernel) and Fabian Abel (from Xing) spoke about recommending jobs to people. While Jakub’s talk focused more on semantic search and  matching, Fabian’s talk revealed Big data technologies behind the recommender engine in Xing. Fabian’s slides are available here.

The third speaker -Rahim Delaviz (from Coolblue)- talked about building a recommender system for an online retailer and the challenges behind such development.

Finally, Anne Schuth (from UvA, CLEF Living Labs) spoke about the importance of online experimentation and he introduced and discussed several online evaluation approaches. The slides of his talks are available here.

The RecSysNL meetups have been quite successful so far – thanks to its initiators: Alan Said and Alejandro Bellogin. This indeed shows how much recommender systems are in great demand in all domains.

This slideshow requires JavaScript.


3rd RecSys:NL Meetup

The 3rd RecSys NL meetup will be on Monday 8 Dec at SDL Amsterdam. There will be four great speakers from Xing, TextKernel, CoolBlue, and UvA (LivingLabs)  talking about their latest challenges and solutions in the area of recommender systems.

  • The meetup will start at 5:30PM sharp.
  • Address: Hoogoorddreef 60, 1101 BE Amsterdam
  • Please note that you need to sign up for a free ticket to participate. Tickets are available at our Eventbrite page.

Talk 1: Recommendation systems for jobs and people (by Jakub Zavrel)

Most systems in HR, recruitment and job search are very transactional because the software does not understand the concepts described in unstructured documents: CVs and job profiles. This transactional nature makes it very difficult to optimize conversion for both the job seeker and the recruiter. Semantic technology, and in particular semantic matching of jobs and people has the potential to significantly improve the user experience and effectiveness for both sides in the labor market. The key enabling technology is NLP-based understanding of unstructured data and the application of domain knowledge to facilitate more relevant recommendations. In this talk I will sketch the most promising use cases, current technology limitations, give an insight into ongoing R&D by Textkernel, and present a number of challenges for the audience.

Speaker Bio:

Jakub Zavrel is founder and CEO of Textkernel, a semantic recruitment technology company based in Amsterdam. With an R&D background in natural language processing, big data, machine learning, and semantic search, he and his team work on building technology for the future of global recruiting. For multi-lingual signal and noise in the latest semantic technologies. You can follow him on twitter @jakubzavrel

Talk 2: Recommending Job Ads to People (By Fabian Abel)

XING is a business network that allows people to find new jobs and other business & career opportunities. In this talk, we will sketch the development of the job recommendation system that serves Millions of XING users with job offers. We will discuss key challenges of building such a recommender system, present key features of the job recommendation algorithms and outline the technical infrastructure on which we operate our recommender systems.

Speaker Bio:

Fabian is a Data Scientist at XING and enjoys working on large-scale data mining problems and delivering data products to large audiences. You can follow him on twitter @fabianabel_

Talk 3: Do’s and not to do’s while building an Industrial Recommendation System (By Rahim Delaviz)

In this presentation Rahim Delaviz will share his experience on building a Recommendation System for an online retailer, CoolBlue. He will cover different aspects of a Recommendation System in an industrial environment, like what should be the focus of the team, how to tackle such a project in smaller steps, how to pick the right tools, techniques and data. Finally, what are the best practices to evaluate a Recommendation System?

Speaker Bio:

Rahim Delaviz was born and grow up in the East-Azerbaijan province of Iran, located on the NW part of the country. He did his BSc. in Software Engineering and MSc. in Mathematical programming at SUT, Tehran-Iran. After a few years of working at Iran, he moved to Sweden and followed a MSc. in Distributed Systems at KTH, Stockholm. In 2009 he started his PhD at TU Delft, Netherlands, with the thesis focused on Distributed Reputation Mechanisms. After graduation in 2013, he joined CoolBlue as a Data Scientist, and since then he has been involved on a number of projects such as RS and Big Data Analytics.

Talk 4: Using users for online evaluation and learning (by Anne Schuth)

Deployed information systems are constantly changing. Collections change, users preferences change but above all, researchers and engineers constantly develop potential improvements to algorithms. It is vital for such systems to constantly measure their performance. In particular because by far most of the potential improvements are likely to actually degrade system performance. Even if they seem to be obviously good ideas. The methodology for measuring system performance has radically changed recently and in industry is now mostly done by observing and interpreting user interactions with the system. In this talk, I will stress the importance of online experimentation and I will discuss several online evaluation approaches. Finally, I will touch on approaches that use these online evaluation methods to automatically improve information systems.

Speaker Bio:

Anne Schuth is a third year PhD candidate at the University of Amsterdam within the Information and Language Processing Systems. Earlier this year, he was research intern for 3 months at Yandex and then for 3 more months at Microsoft Research in Cambridge. His current research focuses on interpreting user interactions with search engines. He is interested in how these interactions can be used for evaluation as well as learning.


2nd RecSys:NL Meetup – Wrapup

The 2nd RecSys meetup was held at Sanoma on April 16th, had three invited speakers and was hosted by Sanoma (in their awesome venue on Hoofddorp).

The three speakers, Julian Ortega from SDL, Joaquin Delgado from Verizon, and Nico Schoonderwoerd all spoke about recommender systems, as used by their companies. While Julian’s focus was on how the recommender systems at SDL are implemented and managed, Joaquin spoke of the similarities between ad targeting and recommendation. Nico presented current.ly, the latest product from PeerRech. current.ly recommends tweets based on news trends.




2nd RecSys:NL Meetup

The 2nd RecSys:NL Meetup in Amsterdam will be hosted by Sanoma on April 16th at 18:00.The meetup will start with a few talks and afterwards there will be time to mingle and have drinks.

Sanoma is located at Capellalaan 65, Hoofddorp, close to Hoofddorp train station. Location info.

If you are coming by car, you will be able to park your car in the underground garage at Sanoma for free.

Please note that you need to sign up for a free ticket to participate. Tickets are available at our Eventbrite page.

Talk 1: Recommendations in SDL Fredhopper – Julian Ortega

SDL Fredhopper is a world leading e-commerce solution designed to help marketers target the right customers with the right items in the right context, optimize the online shopping experience and increase conversion rates, revenue and customer lifetime value. It powers over 300 web sites, in more than 40 languages, more than 50 countries and 12 of the top 100 online business – including the world’s 2nd largest online retailer. In this talk, I will bring SDL Fredhopper into context before looking at what happens in the recommendation space our business point of view as a service provider, while drilling down on a short study case of the events revolving around bringing a new recommender system to life – from idea to production. Finally, I will discuss some external influences beyond the technical scope.

Speaker bio:

Since 2011, Julian Ortega has been working as a software engineer for SDL Fredhopper, where he designs and develops processes to deliver product recommendations through Fredhopper’s merchandizing engine, using different techniques with work focused on developing pragmatic software engineering approaches to address the limitations of predictive models and recommender systems in industrial application domains. Julian is also currently half-way through his master in Computer Science at the VU Amsterdam. Find him on Twitter @jortegac

Talk 2: current.ly – Nico Schoonderwoerd

The current.ly app is PeerReach’s latest product. It recommends the best tweets for news trends, virals and memes. How current.ly gets those trends? From a selected group of recommenders!

Speaker bio:

Nico Schoonderwoerd is a physicist, worked at a hedge fund, disrupted real estate search on the web with miljoenhuizen.nl, and now applies his love for data at PeerReach. PeerReach identifies influential and relevant people and analyses 160 million twitter accounts

Talk 3: Scalable recommender systems and its similarity with advertising systems – Joaquin Delgado

In this presentation I will talk about the design of scalable recommender systems and its similarity with advertising systems. The problem of generating and delivering recommendations of content/products to appropriate audiences and ultimately to individual users at scale is largely similar to the matching problem in computational advertising, specially in the context of dealing with self and cross promotional content. In this analogy with online advertising a display opportunity triggers a recommendation. The actors are the publisher (website/medium/app owner) the advertiser (content owner or promoter), whereas the ads or creatives represent the items being recommended that compete for the display opportunity and may have different monetary value to the actors. To effectively control what is recommended to whom, targeting constraints need to be defined over an attribute space, typically grouped by type (Audience, Content, Context, etc.) where some associated values are not known until decisioning time. In addition to constraints, there are business objectives (e.g. delivery quota) defined by the actors. Both constraints and objectives can be encapsulated into and expressed as campaigns. Finally, there there is the concept of relevance, directly related to users’ response prediction that is computed using the same attribute space used as signals.

As in advertising, recommendation systems require a serving platform where decisioning happens in real-time (few milliseconds) typically selecting an optimal set of items to display to the user from hundreds, sometimes thousands or millions of items. User actions are then taken as feedback and used to learn models that dynamically adjust order to meet business objectives. Most of the targeting and real-time decisioning capability on scalable ad systems have been inspired by information retrieval (IR) techniques, which can be directly applied to recommender systems.

This is a radical departure from the traditional item-based and user-based collaborative filtering approach to recommender systems, which fails to factor-in context, such as time-of-day, geo-location or category of the surrounding content to generate more accurate recommendations. Traditional approaches also fail to recognize that recommendations don’t happen in a vacuum and as such may require the evaluation of business constraints and objectives. All this should be considered when designing and developing commercial recommender/advertising systems.

Speaker Bio:

Joaquin A. Delgado is currently Director of Advertising Technology at Intel Media – OnCue (an Intel Corp subsidiary recently acquired by Verizon Communications), working on disruptive technologies in the Internet T.V. space.  Previous to that he held CTO positions at AdBrite, Lending Club and TripleHop Technologies (acquired by Oracle). He was also Director of Engineering and Sr. Architect Principal at Yahoo! His expertise lies on distributed systems, advertising technology,  machine learning, recommender systems and search. He holds a Ph.D in computer science and artificial intelligence from Nagoya Institute of Technology, Japan.


2nd Amsterdam RecSys Meetup – Postponed

We’re currently working together with Sanoma planning the next RecSys meetup. The event will take place on October 28th at Sanoma in Hoofddorp. The building is next to the Hoofddorp train station and easy accessible by car.

More information about talks and other details will be available in the coming weeks.


The event has been postponed, we will be posting a new date shortly


1st RecSys:NL Meetup – Wrap up

The first RecSys:NL Meetup has taken place! The event was hosted by CWI with a line-up of three great talks by Lukas Vermeer of Booking.com, Jimmy Lin of University of Maryland, and Niels Basjes of bol.com.

This first meetup was an experiment checking how big the interest for a meetup like this would. We’re very happy to say that the interest and attendance by far supassed our expectations. There was a crowd of well above 30 people in the audience and the general interest in topics related to recommender system seemed high. Hopefully meetup was only the first of many recurring events.


The following talks were given at the event:

Recommendations at Booking.com

Lukas Vermeer spoke about recommendations at one of the world’s largest booking websites, what type of recommendations, experiments and data provide recommendations for travelers. Even though there were some fine-grained details that Lukas could not reveal, the talk gave a lot of insight into what the team behind the recommendations at booking are facing in order to provide the best experience for their users.

There are no slides available for this talk.

WTF at Twitter

Jimmy Lin spoke about the Who(m) To Follow (WTF) service at Twitter which he was involved in developing during his extended sabbatical at Twitter (2010-2012). As the name suggests, the service recommends twitter’s users other users to follow.

There are no slides available for this talk.

 Big data at Bol.com

Niels Basjes spoke in great detail about bol.com’s MapReduce-based recommendation systems. In the talk Niels described the development of the service from idea to production system and what challenges were involved.


Pictures from the event