May 28, 2014
Adapted from a presentation with Jean Kaplansky at EDUPUB2 Workshop. Wiley recently announced that EPUB 3 is our standard for ebooks in reflowable format. We started publishing in EPUB 3 as of February 1, 2014, and in this article I’ll explain a little about how we developed the template. Wiley publishes professional, STM, scholarly, and higher education content in a variety of subject areas from accounting to zoology, including the For Dummies brand. Wiley has been distributing one EPUB to most ebook retailers since 2008 for most titles, with no customization for individual devices. I work in Wiley’s Digital Book Services Group, developing ebook stylesheets and specifications and, often, representing Wiley on industry working groups. I have been toying with the EPUB 3 spec since before it was officially released.
In 2012, we did a lot of exploration about what would and wouldn’t work with EPUB 3, however, for a variety of reasons, neither we nor the marketplace were ready to move forward with it
at that time.. In 2013, some great things happened. epubcheck 3.0 and epubcheck 3.0.1 were widely adopted, making it possible to validate almost everywhere. I worked with reading systems developers who had not previously been able to process the most basic EPUB 3 file to target specific issues. One of the greatest challenges was the rendering of the navigation document. In EPUB 3, the navigation document is a content document, not just a hidden file to render an automated contents widget (see Matt Garrish’s article for more). However, it is possible to include elements, such as a list of pages for page-mapping that we don’t want the user to see. Several EPUB 2 reading systems choked on this. I would like think Wiley’s discussions with systems developers and the work of the AAP EPUB 3 Implementation Project had a great effect. We saw some great developments in the second part of 2013, so Wiley concluded that it was time to move forward with EPUB 3.
The plan was to build an EPUB 3 template that supports ALL Wiley book products, could be distributed to ALL Wiley EPUB retailers, including those that use EPUB 3 reading systems, EPUB 2 reading systems, and Kindle, and improves accessibility. This sounds huge, but the approach for all of these requirements is similar. Our first step was partnering with Aptara and the brilliant Jean Kaplansky (this is how great friendships are born). Jean is one of Aptara’s Solutions Architects, expert in structured content architecture and development, including EPUB 3. Jean and I worked together to build a basic template that has already served as the backbone for hundreds of Wiley books.
Reading Systems
The current digital production environment changes daily. The number of reading systems available to our customers today is astounding. As an ebook developer, keeping up with reading system requirements and specifications can be a challenge. Wiley distributes EPUB files to more than 50 retailers, most of whom do not officially support EPUB 3. We could not realistically test on every system as we went along. We focused on key channels for our market and on potential growth. For example, we know several developers are looking to make use of the developments coming from the Readium Foundation, and there several retailers focused on text book sales where we expect to see growth.
We ended up doing testing on 28 platforms, which covered a wide range of devices, operating systems, and SDKs. Jean and I spent a few days scouring systems documentation to refresh our memories and make sure that we were not doing anything that conflicted with any specific system’s criteria.
Basic Requirements, Basic Functionality
Our first step was to overhaul the basic content structure. We fully embraced HTML5 in all its hierarchical glory. HTML5 offers semantic content structure and built-in accessibility. HTML5 also future-proofing and repurposing options, as it is the language of the Open Web Platform. It is also easy to adjust and adapt the template to the needs of individual products. Our next step was to incorporate much of the EPUB 3 structural semantic vocabulary, adding precision to the HTML, increasing accessibility and adding features like page mapping, as well as refining the metadata included in our EPUBs.
Loving HTML5 is not enough to make it work on some of the still-popular EPUB 2 reading systems. There are recommendations in the EPUB 3 specification for making files backward compatible. Including a fallback to the EPUB2-friendly .ncx file in addition to the new navigation document and fallbacks for video to images will go only so far when a reading system does not yet support HTML5 and CSS 3. So, we had to get creative, making sure that using tags such as <aside> worked across the board. We took advantage of the fact that systems that respect or ignore CSS 3’s display:none; generally do the same with HTML5’s <aside> and used them in combination. This creates offset text boxes using <aside> for HTML5-aware reading systems and a CSS media query including a hidden </hr> for the reading systems requiring fallback functionality. We look forward to removing the excess tagging required by the workaround in the future as more devices and apps implement the EPUB 3 HTML5 and CSS 3 profiles.
Adding the Sauce
This leads to the question of CSS. How do you create CSS for this variety of books on this many platforms? In short, keep it simple. Simple does not mean ugly. Simple means easy to adapt – the CSS adapts to different screen sizes easily, and I can also adapt the CSS easily. All the chapter titles are blue, no problem. But if your CSS can’t handle that, perhaps you need to find a better method for writing CSS. All the text boxes contain nested lists, and the third level of nesting needs to be marked with lower-case letters with purple background text, also no problem.
The first step to building CSS with common support across multiple platforms was to take a look at the default properties of the reading systems with which we were working. This basically gave us a clean CSS slate to work off. Adding cascading elements to override the defaults came next. We also included some generic selectors and included some empty selectors that we will fill in as the template becomes more robust. One of the beauties of class values is that the presence of a class value doesn't necessarily require immediate CSS support. Empty selectors won't adversely affect formatting and will wait patiently to be filled in when support is required. The last stage was to incorporate selectors and media queries that are provide specific browser or reading system support.
Testing and Compromises
When working with this many reading systems, it is imperative to test, test, and test again. Jean and I learned that, while emulators can be helpful, they can also be misleading. Sometimes emulators render content completely differently from the devices that will be in the hands of customers. Before you give in to the urge to throw your tablet against a wall because it seems to rendering the page-list within the rendered TOC, make sure you’ve downloaded the latest OS and the latest version of the software you’re checking. Not all reading systems announce updates to software, but many make them available on their websites.
Another issue that comes into play when dealing with multiple platforms and systems is compromise. The EPUB 3 specification is rich and full of promise. The goal was to make the spec completely backward compatible, but increased support for the fallback functions has not been added to some reading systems. In maintaining our stance of delivering one EPUB 3 file to all distribution partners, we had to make some difficult decisions.
Headings and The Outline Algorithm
Jean and I went back and forth about whether it was best to begin each <section> within a content document (usually a chapter) with an <h1> or have only one <h1> per content document. All succeeding elements would begin with <h2> and continue on to <h6>. This is a subject of hot debate in the web community. (See http://www.w3.org/TR/WCAG20-GENERAL/H42.html vs. http://blog.paciellogroup.com/2013/10/html5-document-outline/ for more.) We opted to go with one <h1> per content document because current reading systems support this with little additional CSS, and it is easier to transform from Wiley’s XML markup.
<body epub:type="bodymatter">
<section epub:type="chapter">
<header>
<h1>Chapter 5</h1>
</header>
<section>
<h2><a id="OneHead"></a>Level One Head</h2>
<p>This is where the content goes</p>
</section>
</section>
</body>
Lists and Tables
One of the goals of implementing EPUB 3 was to make Wiley books more accessible. One of the challenges in digital publishing over the years has been some reading systems’ disregard for HTML lists (especially with deep nesting). This required some ill-advised markup such as
<div class="bulletList">•List item</div>
Fortunately, reading systems have stepped up their game, and we can now tag that list item as:
<ul><li>List item</li><ul>
A similar issue occurs with tables, but the rendering issues with these are more widespread. Wiley books often contain large and complex tables. Rendering these tables on three-inch screens doesn’t often go smoothly. There is improvement as the screens get larger, but some tables remain illegible until the reader accesses them on a large screen. This is not an issue exclusive to EPUB but an issue worth considering for publishing any content that will be read in a mobile environment. The argument can be made that authors and editors ought to reconsider whether everything that is tabular in a print context should be tabular in a digital context. This is certainly a conversation we have within Wiley, but there is a significant amount of content that is tabular for a reason - accounting tables are a good example. We concluded that including tables in HTML would create a better user experience on more platforms than including tables as images. Many devices allow for panning and zooming on both tables and images, but HTML is accessible, whereas the image is not. A very large HTML table may present rendering difficulties on some devices, but that same table will be difficult to read if presented as an image. In those circumstances, we include the table as HTML but usually make the table image available on a companion website.
MathML
Wiley publishes many books with math, including professional level finance books, college level textbooks, and advanced mathematics books. HTML5 promoted MathML to a first class citizen in the specification. Therefore EPUB 3 implementations must also treat MathML as a first class citizen going forward.
Unfortunately, the number of browsers and reading systems that support MathML is rather limited. So, we had to make the decision to include math as images. (SVG is a good theoretical alternative but presents its own set of challenges.) However, Wiley already creates MathML for every title, and we do not want to waste the opportunity to embed this in our EPUB files. Our temporary solution is to include the MathML but comment it out. When circumstances, such as limited distribution, allow we publish our books with full MathML.
What does it look like?
Seasonal Sector Trades: 2014 Q2 Strategies by Jeffrey A Hirsch and John L Pearson
Fundamentals of Electric Power Engineering: From Electromagnetics to Power Systems by Massimo Ceralo and Davide Poli
How can you get started in EPUB 3?
It is possible to build one EPUB template for all retailers, but be aware of your priorities and the necessary compromises. Following along with web-based "mobile first" design principles, it is a good idea to design for the lowest common denominator and then add layers of functionality to create progressive enhancement. It is easier to add features and functionality than it is to take away. Be patient and work with systems developers. Get involved in spec development. A lot of the work I did on this began with my involvement in EPUB specification authoring. Knowing how the sausage is made will help you make your own sausage.