Detecting Available Free Space in WebSQL

At Safari, we’re working on some web products that allow offline usage. For various reasons, using the Application Cache isn’t appropriate for our use case; we need a “real” database. The browser requests the data from the server and then inserts it into the relevant tables in WebSQL. (Yes, I know WebSQL is deprecated, but unfortunately iOS doesn’t yet support IndexedDB and this functionality is particularly important on mobile phones, so that’s what we’re focusing on for the moment.)

The Problem

Generally speaking this works well, but you have to navigate the landscape of device capabilities, and there are times when it is particularly frustrating. Take for instance this scenario: I chose some content that has a lot of images, adding up to 18MB of stuff, and decide to save it offline. On iOS, we get 50MB of storage per domain (though the data is actually encoded as UTF-16, so it’s about half that in reality), so this should be OK.

So the content is requested from the server, streamed to the client, and the client attempts to save it in the database. What we didn’t know is that there’s already  12MB worth of stuff stored in the database. So what happens? After making the user wait to download the entire 18MB blob and then wait to parse it and try to load it all in the database (which could amount to a fairly large amount of time depending on a variety of conditions) – the database throws an error and we tell the user they’re out of space. I think you’ll agree that this experience is sub-optimal.

A solution?

Can we do better? I think we can. I want to preface this by saying that this most likely isn’t for everyone. It may very well be faster to just throw your data at the database and see if it sticks, but in our situation most of the time it will be faster to detect and then decide what to do. The problem is that there’s no built-in way to determine how much space is left before you’ve hit whatever cap there may be. Believe me, I googled quite a lot looking for a solution. After that search and chatting with some co-workers, we came up with something and that’s what I want to share: a proof-of-concept I’m calling yardstick.js. I’ve created a gist of the code on Github.

Using it

Yardstick works by using a temporary table to measure how much space is left by filling up to a given targetSize. Originally, I wanted to create a separate database and exploit the per domain storage limitationbut there were some issues with the allocation of the two different databases. Say we are in that situation above, we have already prompted for the max size (see here for why this is important), and we know that the content we want to insert is 18MB, usage of yardstick.js would look like this:

  var ys = new Yardstick();
  ys.measure(18, function(avail, err) {
      // continue what you wanted to do here
      alert(avail + 'MB available for saving');
    }
  );

What’s happening

An in-memory string of 1024 * 1024 characters is used to fill the database using targetSize for the number of loops. So we can assume that each loop inserts 1MB. On each successful iteration we set the internal available attribute to equal the result.insertId of the SQLResult returned, letting the auto-incrementing table do the counting for us. The result of this setup is that if we get to targetSize loops, we know that amount can be inserted, if we error before getting there, we have the internal available attribute that will tell us how much we can safely insert. Upon completion, it deletes the in-memory fake data and drops all of the records that were created, and then calls the passed-in callback with the value of available and the error object, if an error occurred.

Now we can determine if we can save that content locally before we make the (even more) expensive call to the server. And if we can’t save the 18MB, we know how much we can save so that we might be able to suggest saving alternate content that’s smaller.

What do you think?

This is currently just an idea and I’d love to get some feedback on it because I think there are others who know more about this than me. There are obvious downsides to this approach (speed being the obvious one; I also killed the browser on my iPhone 3GS trying to use it, and it probably has battery life implications) and I’m sure there are improvements to be made as well. In addition to the gist, I wired up a little playground. The first input will create a database and fill it with that much dummy data, you can then input a size you’d like to save and it will use yardstick.js to determine if that’s possible. If you have any feedback or want to talk about this, please leave a comment here, on the Github gist, or contact me on twitter at @meirish.

Chaining and Reversing CSS Transitions

CSS3 has given us a lot of goodies that help front-end devs eliminate the use of images to achieve the stylistic effects that so many designers love to use. These include gradients and drop shadows on both text and DOM elements (and soon we’ll have blend modes!). There are also transitions and animations which, in many simple cases, can eliminate the need to use JavaScript to animate, though we still need JS to add or remove classes to trigger those effects.

I was recently reading Alex MacCaw’s great post on CSS Transitions and I learned a lot; you should check it out. I was surprised, though, that he left out mention of the transition-delay property. I’ve found it to be useful for doing some more complex transitions without building out a full JS framework like he does in his post. Let’s walk through how transition-delay can help with this.

The Setup

I’m just going to show the pertinent snippets; obviously you should use whole HTML documents! If you want to play with the examples or even fork them, they’re up on codepen.io. For our example, I’m going to build a drawer that slides out when you click on an anchor link.

The HTML

<div class="drawer">
  <div class="drawer-content">
    Some content here
  </div>
</div>

The CSS

* {
  box-sizing: border-box;
}

body {
  background: #ccc;
}

.toggle-thumb {
  display: block;
  width: 45px;
  position: absolute;
  top: 0;
  bottom: 0;
  right: 0;
  transition: all .2s ease-in;
}
.toggle-thumb:hover {
  background: rgba(255,0,0,.5);
}
.drawer {
  position: fixed;
  top: 0;
  bottom: 0;
  width: 275px;
  left: -235px;
  padding: 40px 45px;
  margin: 0;
  z-index: 20;
  background: rgba(255,255,255, .98);
}

.open.drawer {
  left: 0;
}

Nothing crazy here, just a few divs and our anchor to get everything in place. If you click through, the button is the white bar on the left side of the page. We’re going to open the drawer by adding a class open to the drawer container. In the example, this moves the drawer over so that you can read the contents. Currently there are no transitions, so at this point we’re seeing what a browser that doesn’t support transitions will look like.

The white bar on the left will be the clickable area for the sliding drawer.

The white bar on the left will be the clickable area for the sliding drawer.

View on Codepen.io

Adding a Transition

This one’s easy, we just add a transition. In the example I’m just using the transition shortcut, but chances are you’ll have to add a variety of vendor-prefixes as well.


.drawer {
  position: fixed;
  top: 0;
  bottom: 0;
  width: 275px;
  left: -235px;
  padding: 40px 45px;
  margin: 0;
  z-index: 20;
  background: rgba(255,255,255, .98);
  /* trigger HW acceleration */
  transform: translate3d(0,0,0);
  transition: left .15s ease-in-out .3s;
 }

.open.drawer {
  left: 0;
 }

You’ll notice that we added the transition on the base class, not on the class that we’re adding. This makes sure that the transition happens both when adding and when removing the class. If it were just on the class that was being added, the element would snap back without the transition.

View on Codepen.io

Chaining Transitions

Here’s where transition-delay comes in handy. If you need to transition in a certain order, you can actually delay them. Couple this with the fact that each element can have multiple transitions and you can start to chain transitions. To get our transitions to chain we’re going to use a delay for second transition that’s equal to the duration of the first, that way we’ll get a two transitons back to back just by adding a single class.

.drawer {
 position: fixed;
 top: 0;
 bottom: 0;
 width: 275px;
 left: -235px;
 padding: 40px 45px;
 margin: 0;
 z-index: 20;
 background: rgba(255,255,255, .98);
 /* trigger HW acceleration */
 transform: translate3d(0,0,0);
 transition: box-shadow .2s ease-in-out, left .15s ease-in-out .3s;
}
.open.drawer {
 left: 0;
 box-shadow: 10px 0 15px rgba(0,0,0, .3);
}

Here, we’ve added a second transition for the box-shadow property and we added the styling for the box shadow on the open class because we want to add the shadow as it opens to give the illusion of it coming off of the page.

Clicking on the white bar (pink when hovered) will slide the drawer out using the animation

Clicking on the white bar (pink when hovered) will slide the drawer out using the animation

View on Codepen.io

Getting It to Reverse

You might have noticed in the last example that the reverse case doesn’t exactly work properly. It still transitions the box-shadow first when removing the class. We can fix this by moving our current transition declaration to .open and then adding one to .drawer that swaps the order of the transitions.

.drawer {
 position: fixed;
 top: 0;
 bottom: 0;
 width: 275px;
 left: -235px;
 padding: 40px 45px;
 margin: 0;
 z-index: 20;
 background: rgba(255,255,255, .98);
 /* trigger HW acceleration */
 transform: translate3d(0,0,0);
 transition: left .15s ease-in-out, box-shadow .2s ease-in-out .3s;
}
.open.drawer {
 left: 0;
 box-shadow: 10px 0 15px rgba(0,0,0, .3);
 transition: box-shadow .2s ease-in-out, left .15s ease-in-out .3s;
}

View on Codepen.io

Wrap up

So that’s it! You might ask, “Why use this over animations which (now) have animation-direction built in?” I find transitions a bit more concise and manageable. This technique does lose its advantage over animations, though, when you start having to do more than two transitons. It simply becomes cumbersome, and manually re-ordering transition definitions doesn’t scale well.

Thanks for reading and I hope someone will find this trick as useful as we have.

Choose Your Libraries Wisely

At Safari we use a variety of JavaScript libraries. As we have expanded and started new projects the question: “how do we choose when to use what library?” has come up a few times. I’ll go through my general approach to choosing a JavaScript library and then talk over specifics around the three main libraries we use at Safari: jQuery, Underscore, and Backbone.

Deciding Which Library You Need

Generally you start with a problem. I first want to make an observation about jQuery plugins becuase I think it’s where a lot of people start when looking for a solution (I know it was for me). Many who already use jQuery will just get out their Google hammer and search for “jQuery + [insert problem here]“. There is a jQuery plugin for pretty much anything you’d want, but I think coupling all of your libs could lead to problems later. If all the plugin does is attach a few methods to jQuery.fn, then my advice would be to look for a jQuery-free version. The added benefit of a jQuery-free library is that if you ever decide to switch from jQuery to something else, you’ll have less code to convert or find a replacement for.

OK, so sometimes a jQuery plugin will be your solution, but I want to start earlier in the decision process than that: start by thinking about your problem and ask yourself “is JavaScript the right tool to solve this problem?”. If your answer is “yes”, ask yourself the question again. I’m not trying to be a jerk, it’s just that CSS3 can do a lot of things that used to solely be in the realm of JavaScript. Now it’s better to write a CSS class to do that and then use some lightweight JavaScript to add or remove that class when appropriate. Just because modern browsers can handle most of the JavaScript we can throw at them doesn’t mean our users (and their networks) should have to.

If JavaScript definitely is the answer, then there are a few criteria to use when deciding on a library.

Does it do what you want?

If it doesn’t do what you want, then obviously, you want to keep looking. The other thing to look at here as well is compatibility: will it work in the environments (mobile browsers, desktop browsers, node) you need it to?

Do you like the API?

Read through the documentation (no docs? we’ll hit that one next) and try and get a feel for how the library is used. Do you like the choices that were made during the API design? If you were to using this library on a daily basis, would you enjoy it?

Is there documentation?

While most of us are capable of digging around in the source to figure out what’s going on, it is great when we don’t have to. Just a list of API methods with their corresponding docstrings is often passable, but I prefer documentation that is more user-friendly.

Are there tests?

Having tests is a good sign in a project. It helps to keep the codebase stable and allow others to commit to the project more easily.

Has there been recent activity on the project?

Though this is not always an indication of quality, I’m usually more comfortable using a project that is actively maintained. It makes me feel like bugs in the project will get worked out in a timely fashion.

Deciding When to Use a Library

The three main libraries we use heavily at Safari are jQuery, Underscore.js, and Backbone.js. I think all of them meet the criteria outlined above, but knowing when to use each of them is not always straightforward.

jQuery

For me, jQuery is the tool I reach for whenever I have to do anything that involves

  1. DOM manipulation / traversing
  2. Event management
  3. Ajax anything

Lately I’ve also found jQuery.Deferred to be useful in some cases (and really fun to use). jQuery’s great because of it’s ease of use (I remember my initial reaction to jQuery being something like: “All I need to do to get started is read some documentation and throw around some CSS selectors?!”), and the cross-browser differences it smooths over.

More recently, I’ve been tinkering with the idea of only loading jQuery for older browsers that need compatibility and using something smaller like Zepto or Ender for my DOM needs. Though if this commit or this tweet are any indication, maybe jQuery’s 2.0 will be that smaller alternative.

Underscore.js

Underscore.js provides a bunch of utility methods for dealing with Arrays, Objects, and Functions in a functional programming style. The cases where to use Underscore are:

  1. When you want to use Array extras (Array.foreach, Array.map, etc.) in older browsers.
  2. Function.bind – Underscore’s bind and bindAll are handy in browsers where Function.bind isn’t an option
  3. When you need a lightweight template function. Underscore’s template is a slightly modified version of John Resig’s micro-template function
  4. Many other instances—really it’s just very useful, go take a quick skim through the documentation, there’s a lot there.

I like that Underscore is already pretty small (4kb minified and gzipped according to the site), so I don’t often fret too much about size when I think it might be nice for something I want to do. If you’re size- or performance-obsessed though, you might want to check out John-David Dalton’s lo-dash. You can do custom builds, and the focus is more about cross-browser consistency and performance where Underscore’s is generally more focused on spec compliance.

Backbone.js

Backbone relies on both jQuery and Underscore (or one of the alternatives I mentioned for each of those ie., jQuery can be swapped out for Zepto or Ender and Underscore can be swapped out for Lo-dash), so while it’s fairly small itself (6.3kb), there is a bit of overhead associated with its dependencies.

When do you use Backbone?

  1. Single page applications. These have become popular as of late, if you’re not sure what that phrase means, check out the examples on Backbone’s site – most of them are “single page applications” where Backbone is controlling the routing.
  2. When your code feels like a tangled mess. The first time I used Backbone was because I was looking for a way to organize a bunch of spaghetti jQuery code. Backbone’s abstractions of View, Model, Collection, and Router often fit with things you’re already doing once you get to a certain complexity in a JavaScript application.
  3. If you’re trying to manage a lot of events, Backbone’s View is very handy.
  4. If you need to control and respond to browser history with hash URLs or using pushState, Backbone’s Router is worth looking at.

Backbone is a MV* framework, one of many in JavaScript, but what I like about it is its flexibility and the fact that’s it’s fairly bare-bones (seing a common pattern here?). It’s just enough to do what you want and then get out of your way. Some think this leads to too much boilerplate code, but for me, it lets me do things the way I want.

We have other libraries we use as well, but some combination of these three form the base for most of our projects. They might not work for you and your team, but I highly recommend settling on some standard libraries so that don’t have to learn a new library on each new project.

Touch Input and Responsive Web Design

With the release of Windows 8, touch has finally landed on the desktop. There have been some great posts about the design implications of this change, but I haven’t seen much about technical implications, so I decided to write this. Chances are you’ve thought about  touch interaction if you were building a separate mobile site, but what does it mean for a desktop site or a responsive site?

Why Optimize for Touch?

Why is this something you’d want to do? It boils down to making things better for those who visit your site regardless of the device they’re using. This is why many people and companies have chosen Responsive Web Design. But what does Windows 8 have to do with any of this? It’s the first desktop operating system to include touch (and pen input, but more about that later) as an OS-wide, first-class input mechanism. So suddenly, if you have touch hardware, your desktop browser is going to be firing real touch events instead of using a third-party software abstraction to fire mouse events in the OS.

Who should be checking to see if their site needs any changes? If you’re doing anything with JavaScript click-handlers – most sites these days are doing something - then there are some touch-related things to think about. Additionally, maybe you want to consider gestures of some sort or you want to incorporate two or three-finger touch events. But the most important thing to consider is how you handle the simple single-touch event. Often you’ll want to trigger a touch event instead of click.

I’m going to focus on this last one as I think it’s the most common case. The reason single-touch needs “fixing” is that click events incur a penalty in the form of a 300 millisecond delay on many touch-enabled devices. This is to allow the OS to determine if you meant to pan or zoom instead of just clicking. You can speed up this interaction by checking those things yourself.

Optimizing (allthethings) for Touch

There are a few ways to do this. You can feature-detect touch events and then bind to “touchend” instead of click. (Modernizr is great for this but may be overkill for this one use case.) If you decide to go this route you will have to manage everything yourself: handle touch, determine if the user is scrolling/panning/zooming (if they are: don’t fire the handlers, if they’re not: fire it), and then after you fire it there will still be a click event fired.

What—you were expecting this to be straightforward? We are dealing with web browsers after all! What can we do to make this easier? Use a library, of course! But before doing that, I recommend reading through this Google post (this was the first place I read about the technique) so that you understand everything involved with overriding the default browser behavior. The library I’d recommend using is FTLabs’s FastClick. Be sure to read through the source too, it’s well documented. The nice thing about FastClick is that you only have to bind it once for the whole document and it only inserts itself if your device supports touch, as it has the feature-detection built in.

The Touch Future (or: When Touch is Not Enough)

Touch and click events are closely related as there was some effort to have backwards compatibility. But leave it to IE to break up the party. Turns out IE10 and “Metro” apps don’t have touch events (read more about the problem in this Modernizr issue on Github). Instead, Microsoft has abstracted and unified input mechanisms with what they’re calling pointer events. Instead of binding to touch or click, you bind to pointer event and the OS knows when to fire it based on the input method you’re currently using (touch, mouse, pen, laser, etc.). Boris Smus has written a polyfill for pointer events and has a great post explaining it, though I think it might not be ready for primetime. Recently, the W3C has formed a committee to begin the standardization process for pointer events, so it’s definitely something to keep an eye on.

I think pointer events will be the way forward because they are a better abstraction, and you don’t have to do a bunch of hokey pokey to get things to work properly, you simply let the browser do the heavy lifting for you. Even if that’s true though, touch events aren’t going anywhere anytime soon. They’re something you’ll need to feature-detect (there’s talk of defining Modernizr.pointer or Modernizr.mspointer in the above Modernizr issue) and then do what web developers always do: treat IE differently.