Moliere Software Overhaul

February 28, 2018

Over the last couple of days, I have retooled MOLIERE into a system that anyone1 can deploy it and run their own queries. The code is over at the default repo2 and should be pretty straightforward, the code even downloads raw data itself! Just run and point it at a big parallel file systen — in a few hours you’ll have your very own knowledge network!

This project was a pretty crazy conglomeration of everything I’ve been up to. To start, this implements a new stopword cleaning method (inspired by this list from Swanson). We also use a new phrase mining tool, AutoPhrase, to replace ToPMine.

Overall, these improvements are qualitatively more interpretable than the previous method. And, the changes in the codebase make it easier to add or remove new data sources. For example, this network construction script can choose to include UMLS based on whether the use has it installed. I’m planning on extending this to incorporate SemMedDB and hopefully some gene-specific data.

Also, in the short-term, I’m hoping to redo my validation experiments, as presented in this paper, to see how these changes may have improved our overall performance.

  1. With a big enough super computer. [return]
  2. For anyone keeping up with the development, this code will look familiar to the MOLIERE_QUERY_RUNNER repo. This code is going to be phased out for the newly updated repo. [return]