Most cardiac action potential models are published in a machine-readable markup language called CellML. This is a very good thing, because it unambiguously describes the system of equations that makes up the model, allows people to exchange and use them without the chance of introducing typos every time they translate it to a new language (or indeed publish it in print), and allows you to use whatever software for simulations you care to choose.
Back in 2011, Jonathan Cooper, Steve Niederer and I published a paper that proposed/developed a similar language for describing simulated experiments, in a format so that they could be applied to a wide variety of relevant action potential models.
One of the main reasons for this was to allow us to examine the functions, or behaviours, of the models in certain scenarios, and to compare them against experimental data from the same scenario. We termed this concept “functional curation“.
In the paper we took cardiac electrophysiology models in the CellML repository, ran the same protocols on all of them, and highlighted how quite a few of them failed to replicate some fairly basic electrophysiology properties.
In particular, we examined how the length of time a cell is electrically active (its Action Potential Duration, or APD) depends on how frequently it is stimulated. As you might expect, a cell that is stimulated more frequently needs to be electrically active for a shorter amount of time, to be ready for the next heart beat; whilst it makes sense to be active for longer at slow stimulation rates to avoid being susceptible to ectopic beats. This is known as APD restitution, and studying it on the first beat of a new pacing regime is called ‘S1-S2 restitution‘, as opposed to steady-state restitution.
Experimentally you tend to record a nice smooth ln(x) shaped graph, which would [perhaps?!] suggest some fairly simple underlying mechanism for this behaviour. Anyway, whatever the mechanism, the Beeler-Reuter model from 1977 replicated (qualitatively) this experimental behaviour very nicely:
So we were quite surprised and dismayed to find that a lot of the newer models didn’t do anything nearly so sensible – see the paper for full details. A particularly odd example was the Decker 2009 dog model:
We didn’t think anything of this really, as a lot of the models were and are doing funny things, most of them because the model equations really do behave like that. Fast-forward to 2013 and Yoram Rudy, an author on the Decker 2009 paper, came to visit our group..
I was explaining our work on the functional curation system, and showed him the results of the S1-S2 study. Prof. Rudy was surprised the model performed like that, not least because the title of their paper was “Properties and ionic mechanisms of action potential adaptation, restitution, and accommodation in canine epicardium“! Sure enough we checked the paper, and the results in there were much more sensible than those shown above.
Prof. Rudy suggested that we compared our simulations with the code he publishes on his lab website. So I spent a couple of days tracking down the reason for differences between their code and the model as published on the CellML website. To be fair to the CellML curation process, there was a note in the repository that the file “does not yet recreate the published results – the action potential is close but the calcium dynamics still require some fixing” – something we missed when including it in our paper! To cut a long story short, there were quite a few differences in the end – detailed here on the tracker item to replace the original CellML file, most of them small, but one mistake in a CellML calcium equation that was clearly a typo, and fixing this gave us a curve almost identical to the one in the paper (compare with Figure 3B in their paper, cycle length S1 = 1 second):
So the Decker CellML file has now been updated, and there are a few morals to the story:
- Be careful that the equations you are using are an accurate representation of the model.
- Be careful that the model you are using behaves appropriately for the study you are performing.
- I hope you haven’t been using it for arrhythmia simulations!
We reckon that the best way to prevent these problems is to replicate the results from a paper using a CellML file, and a similar unambiguous description of the protocol that is being applied. Ideally you’d do this for each figure in a paper. To this end we’re proposing a few enhancements to the Simulation Experiment Description Markup Language (SED-ML), that will enable it to do complicated protocols like the above. We’ve called this Functional Curation for now, it is open source and you can play with it if you like. There are lots of future plans, but we’re planning to publish a more systematic comparison of CellML models under a wider range of cardiac protocols on a dedicated website soon.
EDIT: I’ve used the new comments on PubMed to make a note about this next to the paper.