Data Scientists vs. Data Engineers

More and more frequently we see organizations make the mistake of mixing and confusing team roles on a data science or "big data" project - resulting in over-allocation of responsibilities assigned to data scientists. For example, data scientists are often tasked with the role of data engineer leading to a misallocation of human capital. Here the data scientist wastes precious time and energy finding, organizing, cleaning, sorting and moving data. The solution is adding data engineers, among others, to the data science team.

Weather Company New Forecasting Platform

Interesting article on how the Weather Company built a new modern data anlytical platform for both better predictions and better global service using Riak NoSQL database and AWS. The platform (SUN - Storage Utility Network) captures 2.25 billion weather data points15 times per hour.


"As with any large-scale, algorithmic-type modeling, the more data you have, the better the predictions will be..." 

Using AWS to Build a Graph-Based Product Recommendation System

Magazine Luiza, one of the largest retail chains in Brazil, developed an in-house product recommendation system, built on top of a large knowledge Graph. AWS resources like Amazon EC2, Amazon SQS, Amazon ElastiCache and others made it possible for them to scale from a very small dataset to a huge Cassandra cluster. By improving their big data processing algorithms on their in-house solution built on AWS, they improved their conversion rates on revenue by more than 25 percent compared to market solutions they had used in the past.

LinkedIn DSA credential

The Data Science Association now has a LinkedIn group for DSA members only. To join the group, first become a member of the Data Science Association here on this website, and then apply to enter the LinkedIn group over on LinkedIn by supplying your username or e-mail there in your request.