Apache Pinot is an open-source analytics platform that utilizes an OLAP database to provide low-latency insights into large amounts of data. OLAP stands for Online Analytical Processing and is a method in which data from multiple sources can be used together, allowing companies to group data from websites, applications, internal systems, and more together for … continue reading
LinkedIn has announced it is open sourcing its control plane for managing tables in data lakehouse deployments. The tool, called OpenHouse, has been in use at LinkedIn for the past year. The company has 3,500 OpenHouse tables in production currently. It was designed to offer self-service management of tables in open data lakehouses. According to … continue reading
LinkedIn recently announced its decision to open source its Developer Productivity and Happiness (DPH) Framework. The DPH Framework describes “the systems, processes, metrics, and feedback systems we use to understand our developers and their needs internally at LinkedIn,” Max Kanat-Alexander, principal staff software engineer at LinkedIn, and Grant Jenks, senior staff software engineer at LinkedIn, … continue reading
Earlier this week, LinkedIn announced it was open-sourcing AvroTensorDataset, which is a “TensorFlow dataset for reading, parsing, and processing Avro data.” Apache Avro is the primary storage format that LinkedIn uses for its training data. According to LinkedIn, it was experiencing bottlenecks in its machine learning workloads that were caused by the need to read … continue reading
LinkedIn has announced that the most used, core aspects of Feathr, its feature store for productive machine learning (ML), are being open sourced. Feathr works to address the problem of overburdening teams with the increasing costs of maintaining their feature preparation pipelines. Feathr works as an abstraction layer that provides users with a common feature … continue reading
FastTreeSHAP is a Python package that enables the efficient interpretation of tree-based machine learning models by computing sample-level feature importance values.2 The project was recently open-sourced by LinkedIn and was used at the company to improve member experience in products such as People You May Know (PYMK), newsfeed ranking, search, and job recommendations, as well … continue reading
Microsoft’s next step in its global skills initiative is to help 250,000 companies make a skills-based hire in 2021 together with LinkedIn. The company launched its global skills initiative last summer and has since helped over 30 million people learn new digital skills. “A new generation of 21st century infrastructure calls for new investments that … continue reading
Earlier this week LinkedIn announced the open sourcing of Smart Argument Suite, a new Python tool designed to help users pass arguments through the command line interface and consume them in a “human-friendly” way. According to the company, while there are plenty of open source projects that offer CLI argument parsing, they don’t deal with … continue reading
LinkedIn wants to address bias in large-scale AI apps. The company introduced the LinkedIn Fairness Toolkit (LiFT) and shared the methodology it developed to detect and monitor bias in AI-driven products. LiFT is a Scala/Spark library that enables the measurement of fairness, according to a multitude of fairness definitions, in large-scale machine learning workflows. It … continue reading
Microsoft and LinkedIn have launched an initiative to bring digital skills to 25 million people by the end of year. The initiative will focus on the use of data to identify in-demand jobs and the skills needed to fill them. It will provide free access to learning paths and content, as well as low-cost certifications. … continue reading
Unsatisfied with the available solutions for connecting the analytics-generating power of their TensorFlow machine learning implementations with the scalable data computation and storage capabilities of their Apache Hadoop clusters, developers at LinkedIn decided that they’d take matters into their own hands with the development of this week’s highlighted project, TonY. “TonY stands for TensorFlow on … continue reading
As artificial intelligence becomes more popular, skills like machine learning and deep learning are becoming more in demand. New data released from LinkedIn shows job recruiters are fighting to find machine learning talent. “Machine learning is one of the most in-demand skills. It can feel like everyone’s competing for the same small talent pool, especially … continue reading
