cdp/progress20071217

Dec 17, 2007 Progress Meeting

Back to Reportable Condition Processor

Attendees: Shaun Grannis, John Brown, John Hook, Shahid Khokhar, Jeff Friedlin, Dave Trepanier (ISDH), Ben Tuttle (ISDH), John ?? (ISDH)

1. Database bottlenecks are being observed. The current version of postgres on Aslan is 7.4. Shaun and James Egg will investigate installing postgres 8.X. We will report our findings in one week. If upgrade can be performed, aim to complete by next meeting in two weeks.

Status: complete. James installed Postgres version 8.1 on Aslan and John be successfully ported over to this version.

2. John B noted a minor potential bug (that has not yet been encountered) with the current JVM on Aslan. Shaun and James Egg will review upgrading Java and Aslan. Will review findings that next meeting.

Status: ongoing. We will monitor expectantly for any incompatibilities.

3. John Brown will add HL7 message processing queue IDs to the logging facility and also will ensure the MPQ ID's and MSH segments are included in the HL7 messages stored in the tblRawHL7 table. This is necessary for comparing old to new and assessing accuracy.

Status: complete.

4. The LOINC frequency process is inefficient. Shaun and James will look at upgrading postgres on Aslan (#1 above). If we cannot improve DB performance, John B will consider a workaround, such as caching strategies. If time permits, John B may evaluate DB versus caching strategies on his local machine and report out at next meeting.

Status: complete. John B implemented in-memory caching of the LOINC frequency table. This resulted in a nearly tenfold increase in performance, processing approximately 50,000 messages per hour, versus 6,000 per hour previously. This in-memory caching implementation will require that the LOINC frequency query must "roll-up" some duplicate LOINC counts.

5. Mark Tucker will work with John Brown to create functionality to invoke John Brown's CDP by next meeting in two weeks.

Status: ongoing. Mark will work with John be to implement functionality to invoke John Brown's CDP.

6. Mark Tucker will investigate setting up a wiki to track CDP development and will also investigate feasibility of adding discussion list technology by next meeting.

Status: complete. John Hook pointed us to the SourceHub site on Aurora. We can begin tracking progress there. Additionally it was noted that the CDP source was also available in a subversion repository on Aurora. John B can provide a link to the Subversion repository.

7. Shaun and Jeff F will evaluate the new communicable disease processor output and compare it to the old and report the findings at the next meeting in two weeks. Shaun will work with John B to ensure that all data needed for analysis is available.

Status: ongoing. Jeff noticed that the test to reportable condition mappings were incorrect. John B corrected the mappings and re-ran the remainder of the ~700,000 test messages. Jeff's analysis is Here. The old ("current") processor found ~2,000 reportables for the same set. This "new" processor currently only looks for exact match strings -- no string nearness is currently implemented.

8. After reviewing the next set of reportable results messages, Regenstrief (Shaun) will send the ISDH a batch of sample messages for their review and comment.

Status: ongoing. Awaiting results messages.

9. Public health is interested in "denominator data" (LOINC counts by source by day), which we track in the LOINC frequency table. Currently we do not track ZIP codes in the LOINC frequency tables. John B will add ZIP code to the LOINC frequency table so we can track denominator data by location.

Status: ongoing.

10. To Do List (Not in any particular order):

  • After a performance issues are resolved, we'll establish a real-time feed to the communicable disease processor (Task 5).
  • Build a monitoring system to look for anomalies in the message flow, (eg, the lack of Shigella from Clarian)
  • Identify reporting requirements such as sending aggregate e-mails (as is currently done), delivering HL7 messages to local and state health departments, and potentially delivering results through D4D.
  • Add regular expression (RegEx) functionality to the CDP.
  • Enhancing reportable result with additional data (eg, adding Liver Enzymes ALT and AST to Hepatitis cases)
  • Discuss handing HL7 message exceptions (HAPI related?) -- 1.5% of a recent test exceptioned out.

Back to Reportable Condition Processor

Attachments