The State of RT-Quantitative PCR: Firsthand Observations of Implementation of Minimum Information for the Publication of Quantitative Real-Time PCR Experiments (MIQE)

The application of inconsistent experimental design and techniques to quantitative PCR (qPCR) experiments has resulted in the publication of artifactual qPCR data with potentially misleading conclusions [Bustin 2010; Bustin et al. 2009b], leading to the retraction of high-profile papers [Böhlenius et al. 2007; Retraction, 2010]. This situation is further revealed in the Materials and Methods sections of many publications, where it is evident that primers and/or probes were not validated or the associated sequences were not reviewed for competing sequence homology [Wang et al., 2012]. Finally, a large number of published articles with findings that hinge on reverse transcription (RT)-qPCR data report that normalization was performed using a single reference gene untested for stability such as GAPDH, β-actin, tubulin or 18S RNA [Barber et al., 2005; Jacob et al., 2013; Rhinn et al., 2008; Schmittgen and Zakrajsek, 2000; Thellin et al., 1999; Yang et al., 2012].

Teaching and practicing qPCR according to a well-defined methodology that will ensure quality data has been a central theme in recent years and especially since the inception of the ‘minimum information for the publication of quantitative real-time PCR experiments' (MIQE) guidelines [Taylor et al., 2010]. Adherence to key components of a robust experimental design, including best practices in sample preparation, extraction and storage, RNA isolation and purification, RT, qPCR with validated primers and normalization with stable reference targets, will eliminate erroneous data.

To address the challenge of obtaining precise, reproducible and accurate results from qPCR experiments, a group of scientists came together in 2009 to develop a set of guidelines known as MIQE [Bustin et al., 2009a]. MIQE guidelines were the community's first attempt to map out the methodology and key validation criteria required to perform qPCR experiments. Since their publication, a number of papers have supported the need for a consistent and rigorous methodology to ensure the publication of accurate results [De Keyser et al., 2013; Dooms et al., 2013; Lanoix et al., 2012; Taylor et al., 2010, 2011].

Given their framework for generating robust qPCR data, it is surprising that the MIQE standards have not been embraced more widely in practice. Since 2010, more than 23,000 papers featuring qPCR data have been published, but only approximately 5% of these cite the MIQE guidelines (Google Scholar search for ‘qPCR' after 2010). This low citation rate suggests that the vast majority of labs have either not been informed of the guidelines, have chosen to ignore them or believe that they do not apply to their experiments based on historical knowledge and biases that may date back to the early days of the technique.

The best practices for gene expression experiments as outlined by the MIQE criteria provide a simple and practical road map for scientists to navigate through the design of RT-qPCR experiments to obtain the highest-quality data and avoid common pitfalls in experimental design and execution (table 1). Alternatively, by skipping key steps from the MIQE guidelines, data will likely still be generated but can result in irreproducible and incorrect conclusions (table 2).

The methods for cell and tissue culture sample extraction may vary significantly from lab to lab. With adherent cells, some groups first trypsinize, scrape the plate and transfer the cells to tubes. This is followed by centrifugation to pellet the cells and RNA extraction. Other researchers wash the cells on the plate, add RNA extraction buffer directly to the plate and then scrape the plate to form a stable homogenate. For tissue samples, some labs surgically remove tissue from animals and weigh the samples at room temperature. They then slice the tissue and transfer it into tubes, all at room temperature, and finally freeze the samples at -80°C. Other labs immediately flash-freeze the animal tissue in liquid nitrogen, transfer it into tubes on dry ice and store it at -80°C. These different sample extraction and storage methods can yield vastly different results because the transcriptome is affected by each sample manipulation and can change very quickly in response to chemical and environmental treatments [Huang et al., 2013; Viertler et al., 2012]. Rigorous and reproducible methodology is achieved by halting transcription as soon as possible after sample collection. This ensures that differences recorded between bioreplicates in response to experimental treatments are due to treatment and not an artifact of sample handling.

Keeping tissue frozen until homogenization in a solution containing RNase inhibitors ensures consistent results by preventing inconsistent thawing of samples, which leads to differential RNA degradation [Botling et al., 2009; Huang et al., 2013; Kirschner et al., 2013]. Some labs remove tissue samples from -80°C storage, transport them on blue ice and then proceed to homogenize samples and extract the RNA using a wide variety of techniques, during which the samples may begin to thaw prior to RNA extraction. Others place the sample tubes on dry ice and then grind the tissue to a powder in a mortar under liquid nitrogen before adding the RNA extraction buffer. There are many methodologies, reagents and instrument technologies for tissue disruption and homogenization for both protein and nucleic acid extraction. The goal is to convert the sample into a uniform, stable homogenate in a highly reproducible manner while preventing as much as possible any degradation and transcriptional changes from the -80°C freezer through homogenization.

After RNA extraction, most labs measure the optical density (OD)_260/280 and OD_260/230 to quantify the amount of total RNA and to ensure the sample meets the minimal purity criteria with respect to protein and chemical contamination (minimum acceptable OD values of 1.8 and 2.0, respectively). Samples with lower OD values typically contain higher levels of contaminants that can inhibit both the RT and qPCR reactions, resulting in artificially high and variable quantification cycle (Cq) values and imprecise quantification. In our experience, a test that most labs do not perform is an RNA quality assessment to ensure that samples are not degraded, as RNA degradation can occur even when the utmost care is taken with sample handling [Huang et al., 2013]. RNA sample quality can be measured by visualizing extracted fragment sizes on a denaturing formaldehyde-agarose gel or by using more sensitive and precise instrumentation such as the Experion™ automated electrophoresis system from Bio-Rad or the Bioanalyzer™ from Agilent. As with purity, RNA sample quality is directly correlated with altered Cq values, where a degraded sample can give significantly higher Cq values than an intact sample [Huang et al., 2013; Taylor et al., 2011]. Furthermore, RNA quality has been shown to directly affect reference gene variability and the significance of differential gene expression data [Vermeulen et al., 2011].

The RT reaction is a key step in sample processing. RT priming strategy, dynamic range and RT enzyme type are all important to ensure mRNA expression levels are accurately represented in the resulting cDNA [Jacob et al., 2013]. Transcription of both low- and high-expression targets, and thus a wide linear dynamic range for the RT step, is required for accurate representation of these expression levels in the final data. Performing a serial dilution of the input RNA to determine the linear dynamic range of reverse transcribed cDNA will reveal the amount of RNA required for the RT step to ensure consistent coverage of all targets in the sample. Care should be taken to normalize the amount of input RNA for RT with consistency in kit selection and protocol to ensure that all RNA samples are treated similarly. If different amounts of input RNA are used between samples, variable levels of contaminants can be introduced that may inhibit the RT reaction in an unpredictable manner, resulting in variable RNA coverage and cDNA output. The resulting gene expression results are often uninterpretable; therefore, care should be taken to ensure consistent loading of RNA. The hallmarks of a good RT kit include a mix of random hexamers and oligo dTs to obtain the best coverage of the RNA with high fidelity and robust reverse transcriptase containing RNase H to digest the copied template as the transcript is transcribed. A single-step kit in which RNA is added to a single RT mix containing a combination of the RT and hot-start qPCR mixes in one reaction can help minimize technical variability among samples.

Many researchers do not validate their primers because the sequences were sourced directly from peer-reviewed literature, obtained from prior lab members or directly from vendors as ‘off-the-shelf' assays. This practice presumes that the scientists who originally published the qPCR data correctly validated their primers in the same cells and/or tissues as in the current study set, but this may not be the case [Wang et al., 2012]. Many ‘off-the-shelf' assays have only been designed in silico and are often not provided with any validation data, which may preclude MIQE compliance [Bustin et al., 2011]. Rather than make this presumption, labs should validate all primers - including those used by previous authors and vendors - for primer concentration, annealing temperature, specificity and efficiency, with further validation for linear dynamic range with a standard curve from a representative sample [Mikeska and Dobrovic, 2009; Taylor et al., 2010, 2011]. Ideally, validation should be performed on a qPCR instrument that is enabled with thermal gradient capability and has the sensitivity to detect 10 or fewer copies of the target sequence. The ramifications of poorly validated primers for annealing temperature and efficiency with standard curves are inaccurate Cq values and gene expression results leading to incorrect and even opposite conclusions [Opel et al., 2010]. Because primers require independent validation for each sample type (for example, brain versus heart tissue) and also for each RNA extraction method (such as TRIzol versus a kit-based method), an 8-point standard curve of the appropriate fold dilution of cDNA using a good-quality qPCR supermix is recommended. Only thorough primer validation will ensure that the qPCR reaction conditions are optimal for a given sample set and that the samples are diluted such that reaction efficiency is optimal for each target.

Since the release of the MIQE guidelines, a number of published articles have described the effect of improper reference gene selection on the final data [Barber et al., 2005; Jacob et al., 2013; Lanoix et al., 2012; Rhinn et al., 2008; Schmittgen and Zakrajsek, 2000; Taylor et al., 2011; Thellin et al., 1999; Yang et al., 2012]. Normalization of qPCR data with a poorly selected reference gene can dramatically alter the final results to the extent that opposite conclusions can be obtained when compared to results with normalization with stable reference genes. Many labs performing qPCR on a regular basis have only normalized samples to a single, unvalidated reference gene that they have used for all qPCR projects over many years. The list of potential reference genes has increasingly been chosen from publications referring to the tools GeNorm and NormFinder for reference gene stability. A compilation of at least 6-10 separate reference gene candidate primer pairs should be validated as described in the previous section and then tested for stability in samples derived from each of the experimental conditions using GeNorm, NormFinder and BestKeeper [Jacob et al., 2013; Lanoix et al., 2012]. The result of poor reference gene selection for the final data and conclusions is now well documented and has called into question the validity of publications that have only used a single unvalidated reference gene [Barber et al., 2005; Jacob et al., 2013; Rhinn et al., 2008; Schmittgen and Zakrajsek, 2000; Thellin et al., 1999; Williams, 2012; Yang et al., 2012].

Although some labs continue to argue that MIQE criteria are simply ‘guidelines' that do not necessarily need to be followed, there are very good reasons to adopt the best practices outlined here as well as other elements of the guidelines. While some forethought is required for planning an MIQE-guided experiment, the benefits of following these guidelines help ensure robust, reliable and reproducible gene expression results for publication and provide the confidence that the data, interpretations and conclusions will hold up to reader scrutiny.