Earlier this week LinkedIn announced the open sourcing of Smart Argument Suite, a new Python tool designed to help users pass arguments through the command line interface and consume them in a “human-friendly” way. According to the company, while there are plenty of open source projects that offer CLI argument parsing, they don’t deal with the producer side.

“Passing the arguments through the CLI becomes a producer and consumer problem: on the workflow generation side, you need to produce a set of arguments which are passed to the CLI to launch the jobs; on the other side, the launched jobs would consume the arguments passed from the CLI,” Jun Jia and Alice Wu, senior staff software engineers at LinkedIn, wrote in a post.

The principles of the Smart Argument Suite are:

  • It should be simple. The suite makes it as easy as defining an argument container object and passing it through a function call. 
  • It should be safe. The tool has a verifiable and testable systematic process with certain safety guarantees.
  • It should be human-friendly. According to the team, the tool should be easy for humans to inspect or debug on the serialized form. 
  • It should be extensible. Users should be able to extend the support to the argument container classes if desired. 

“It’s a very common scenario that an AI solution involves composing different jobs, such as data processing and model training or evaluation, into workflows and then submitting them to an orchestration engine for execution. At large companies such as LinkedIn, there may be hundreds of thousands of such executions per day, submitted and executed by multiple teams and engineers,” Jia and Wu wrote. “Any improvements in the tools used by machine learning engineers lead to significant improvements in productivity, which highlights the need for robust productivity infrastructure to support machine learning engineers.”

The solution has been released to PyPl and tested with LinkedIn’s other open-source AI solutions including GDMIx and DeText. 

Going forward, the team plans to add escaping to make the serialization safer and expand beyond the language boundaries.