Technetra

Countdown 2.0: Supporting HTML5 Cache

radkins — Mon, 17 Aug 2009 23:08:40 +0000

Taking advantage of HTML5 caching in an iPhone Webapp can eliminate the need to employ cumbersome data URLs for standalone or offline style applications. The offline storage facilities of HTML5 have been around for the iPhone since OS 2.1 and can, for devices running recent OS releases, significantly reduce the complexity of deploying offline Web applications. Caching also provides the potential for automatic refresh when the standalone Webapp comes back online.

Figure 1: Main Screen (in Full-Screen Mode)

Figure 2: Settings Screen (Full-Screen)

Figure 3: SpinningWheel Date Picker (Full-Screen)

So we “trained” the network connected version of our Countdown Webapp to use HTML5 caching (it already used HTML5 localStorage facilities). The “training” steps were very simple:

Create a manifest which lists all the resources needed by the Web application to work offline. This includes the JavaScript, CSS and image files referenced by the application and its included packages (iUI and SpinningWheel). See Listing 1.
Attach the URL of this manifest to the manifest attribute of the Webapp’s HTML tag. See Listing 2.
Ensure that when the manifest resource is retrieved by the browser, the manifest is served as a text/cache-manifest mime-type by the HTTP server. In our case this meant adding
```
text/cache-manifest manifest
```
to our Apache mime.types configuration file. This is necessary for the moment since the listing of standard media types from IANA, as incorporated by Apache 2.2+, does not yet include this mime-type by default.

In addition, since Countdown is designed to be a standalone application, we added an apple-mobile-web-app-capable meta tag key in order to run in full-screen mode, removing the standard Safari URL and Button bars when launched from the Home Screen.

Countdown 2.0

You can try out this new version of Countdown with your iPhone. Be sure to add the application to your Home Screen using the bookmark button at the bottom of the initial Safari screen. This is necessary in order actually to view the Webapp in full-screen mode. The Webapp may be partly functional under some other browsers like Firefox and Chrome (no support for the SpinningWheel date picker).

You can download the new project files from here.

A big thanks to Sean Gilligan (co-maintainer of the iUI Framework) for suggesting these improvements to Countdown Webapp.

Listing 1: Cache Manifest for Countdown iPhone Webapp (countdown.manifest)

CACHE MANIFEST
iui/iui.css
iui/iui.js
spinningwheel/spinningwheel.css
spinningwheel/spinningwheel.js
countdown.css
countdown.js
countdown_touch_icon.png
iui/listArrowSel.png
iui/loading.gif
iui/selection.png
iui/toolbar.png
iui/toolButton.png
iui/blueButton.png
iui/backButton.png
iui/whiteButton.png
iui/redButton.png
iui/grayButton.png
iui/listGroup.png
iui/listArrow.png
iui/pinstripes.png
iui/toggle.png
iui/toggleOn.png
iui/thumb.png
spinningwheel/sw-header.png
spinningwheel/sw-button-cancel.png
spinningwheel/sw-button-done.png
spinningwheel/sw-slot-border.png
spinningwheel/sw-alpha.png

Listing 2: Adding manifest URL to manifest attribute of HTML tag & removing top and bottom banners with `apple-mobile-web-app-capable` META tag (countdown.html)

...
<html manifest="countdown.manifest">

  <head>
    <title>Countdown</title>
    <link rel="apple-touch-icon" href="countdown_touch_icon.png" />

    <meta name="viewport"
      content="width=device-width; initial-scale=1.0; maximum-scale=1.0; user-scalable=0;" />
    <meta name="apple-mobile-web-app-capable" content="yes" />
...

Countdown iPhone Webapp

radkins — Mon, 10 Aug 2009 23:55:31 +0000

What is it?

Countdown is an iPhone app started at this year’s iPhoneDevCamp 3 which was held at Yahoo! from July 31st through August 2nd.

Countdown is a simple multi-color timer that charts how many days, hours, minutes and seconds are left until a selected target event arrives. For example, this may be a fun (or frustrating) way to count how long you must wait until your next birthday. Built-in events also include “Midnight” and “New Year’s”. There is a “Settings” screen that allows you to configure some of the parameters of the program. One of the most interesting configuration elements is a date picker that uses the hardware accelerated “SpinningWheel” from cubiq.org.

Figure 1: Main Screen

Figure 2: Settings Screen

Figure 3: SpinningWheel Date Picker

What does the app do?

Countdown illustrates

Webkit techniques to build a “native-look-and-feel” iPhone Webapp using JavaScript and CSS,
the latest stable iUI (iui-0.30), and
integration of the latest (updated) SpinningWheel, a powerful JavaScript tool that resembles the native Picker View widget (UIPickerView) on the iPhone. The SpinningWheel tool provides a fast and flexible user interface for many kinds of data input and visualization. It allows multiple columns or slots (we are using 3) to be defined to represent complex values like dates and other tabular data. Interaction with the widget is fully hardware accelerated.

Also, we wanted to create a version of our webapp which could be installed as a stand-alone application that would not require access to a network to run. This means that all data items, including images, stylesheets and scripts, must be bundled with the application beforehand. Dynamic data, such as user-updated application settings, must be kept in local storage by the application across multiple launches and even across power cycles.

What issues did we face?

There were some challenges:

Integration of iUI and SpinningWheel:
Integrating iUI with other packages requires some understanding of how iUI manages HTML containers used for displaying successive screens on the iPhone. Essentially the CSS rules of iUI turn off any unselected, top level elements within the body of an HTML page. In order to get SpinningWheel to work properly with iUI, our application needed to define a CSS rule to ensure display of the dynamically created primary container of the SpinningWheel widget. This container has the id sw-wrapper and is added as a child of the screen’s body tag when the widget is opened. A short CSS rule in our application’s stylesheet (loaded after the iUI stylesheet) was enough to override iUI’s default behavior.
```
#sw-wrapper { display:block; }
```
This simple addition to our stylesheet enabled SpinningWheel to be used with iUI without any modification to either package.

In addition, we had to reposition the SpinningWheel’s view frame, sw-frame for our application’s layout. The values for portrait and landscape orientations that worked for Countdown were
```
body[orient="portrait"] #sw-frame { bottom:112px; }
body[orient="landscape"] #sw-frame { bottom:8px; }
```
Using SpinningWheel with its hardware accelerated animations is a lot of fun and is a great way for a user to see and select tabular data like dates.
Client-side storage using HTML5:
We used WebKit’s support for the localStorage DOM attribute of HTML5. This is W3C’s new Web Storage abstraction for JavaScript managed user data. localStorage supports persistence beyond the current session and is ideal (essential in the case of the stand-alone version of our application) for saving the application’s settings in a server independent manner. On the iPhone, the process of creating the stand-alone application changes the application’s domain origin to a null value. This value is then automatically used as the application’s Web Storage domain for all localStorage access, creating referential consistency.
Converting Countdown into a stand-alone application:
[ Note: the following section applies to this version of the Countdown webapp which was implemented using Data URLs. HTML5 Caching provides a better implementation strategy and is covered in the follow-on article "Countdown 2.0: Supporting HTML5 Cache" ]

Converting this webapp, when implemented using Data URLs, into a stand-alone application required modification of iUI in three ways. First, iUI navigates back to screens presented earlier by updating the application’s global window location attribute with URLs from its page history cache. Due to security constraints, updating the location attribute is not allowed by Safari when running in stand-alone mode. Second, iUI uses AJAX requests to update application content. This is also forbidden by the security context imposed on a stand-alone application. This functionality can therefore be removed to save space. Third, iUI preloads images to improve performance. This is not needed in a stand-alone application.

To address the first of these three issues, the standalone version of our application uses an iUI package modified to allow forward and backward navigation via document section ids (using the link’s hash attribute in same way the unmodified iUI does currently for forward navigation). Countdown has only two working screens, but applications that require more screens would need to enhance this approach with a simple history queue.

To prevent violation of the stand-alone security context, the predefined AJAX GET and POST requests were removed from iUI. This means that all resources needed by the stand-alone application have to be bundled with the program when it is written. This is a constraint that all stand-alone webapps on the iPhone are burdened with anyway, so the need to remove AJAX from iUI was not surprising.

Avoiding preloading of images required only simple changes to iUI’s JavaScript and CSS.

Finally, converting this webapp into a stand-alone application also required constructing a simple installer. Fortunately, we were able to reuse the installer tools discussed in our earlier article “How to Deploy an iPhone Web Application”.

It is important to note that these changes to iUI were only required for building the stand-alone version of our application. For the network connected version of Countdown, both SpinningWheel and iUI could be used without any modifications.

Try Out Countdown!

[ Please check out the new version of Countdown that uses HTML5 caching instead of Data URLs ]

There are two versions of the Countdown application.

Network connected version

Try out Countdown on your iPhone.

This network connected version of Countdown may also work in some other browsers (Safari 4+, Firefox 2+, Chrome 3.0.197+) with limited functionality. [EXPERIMENTAL]

Stand-alone version

Install Countdown as a “native application” on your iPhone Home Screen.

This installable version of Countdown may also work as a bookmark (Data URL) in some other browsers (in particular Safari 4+) with limited functionality. It does not work properly in any version of Firefox. [EXPERIMENTAL]

Download the complete Countdown application and its resources, including the stand-alone and non-stand-alone application versions.

Note that Countdown works best on a real iPhone or simulator. However, the application can be run in a limited way on various modern browsers like Safari 4+, Firefox 3+ and Chrome/Chromium 3.0.197+. In these cases, instead of using the (iPhone specific) SpinningWheel interface, Countdown presents a standard select element for date picking. Countdown also uses persistent storage when the browser (e.g., Firefox 3.5, Safari 4+) supports Web Storage. Otherwise Countdown uses a private array which is recreated for each new instance of the application.

Listing 1: Countdown HTML – Network connected version (countdown.html)


  "http://www.w3.org/tr/xhtml1/DTD/xhtml1-transitional.dtd">

<html>
<head>
  <title>Countdown</title>
  <link rel="apple-touch-icon" href="TOUCH_ICON" />
  <meta name="viewport" content="width=device-width; initial-scale=1.0; maximum-scale=1.0; user-scalable=0;" />
  <link rel="stylesheet" href="iui/iui.css" type="text/css" media="all" /> 
  <link rel="stylesheet" href="spinningwheel/spinningwheel.css" type="text/css" media="all" /> 
  <link rel="stylesheet" href="countdown.css" type="text/css" media="all" /> 
  <script type="text/javascript" src="iui/iui.js"></script> 
  <script type="text/javascript" src="countdown.js"></script> 
  <script type="text/javascript" src="spinningwheel/spinningwheel.js"></script> 
</head>
<body>
 

<div class="toolbar">
  <h1 id="pageTitle"></h1>
  <a class="button" id="backButton" href="#" onClick="Countdown.turnActionButtonOn('settingsButton','doMain');"></a>
  <a class="button" id='settingsButton' href="#setupForm" onClick="Countdown.turnActionButtonOff(this,'doSetup');">Settings</a>
</div>
<span selected="true">
  <h1 class="ctr">
    <span id="main_subtitles">
      <span>Today: <span id="today"> </span><br /><span class="divider" id="divider"> ♦</span></span>
      <span>Start Date: <span id="start_date"> </span><span class="divider"> ♦</span></span>
      <span>Event Date: <span id="target_date"> </span></span>
    </span>
  </h1> 
  <div class="unit" id="countdown_box"> 
    <div id="days_box">
      <span id="days"></span><span class="unit_label">Days</span>
    </div>
    <div class="overlay" id="days_overlay"></div>
    <div id="hours_box">
      <span id="hours"></span><span class="unit_label">Hours</span>
    </div>
    <div class="overlay" id="hours_overlay"></div>
    <div id="minutes_box">
      <span id="minutes" style=""></span><span class="unit_label">Minutes</span>
    </div>
    <div class="overlay" id="minutes_overlay"></div>
    <div id="seconds_box">
      <span id="seconds"></span><span class="unit_label">Seconds</span>
    </div>
    <div class="overlay" id="seconds_overlay"></div>
  </div> 
  <div id="main_footer">
    <div id='update_interval_container' >(updated every <span id='update_interval_text'></span>)</div>
  </div>
  <br /><br /><br /><br /><br /> 
</span>

<div id="setupForm" title="Settings" class="panel">
  <h2>Target Event</h2>
  <fieldset>
  <div class=row>
    <label>Title</label>
    <input id="title_input_field" onChange="Countdown.setTitleFieldResources(this.value);" type="text" size=30 value="" />
  </div>
  <div class=row>
    <label>Event</label>
    <select id="select_event" onChange="Countdown.updateEventType(this);" style="font-size:1.0em; text-align:left;">
      <option value='newyears'>New Years</option>
      <option value='birthday'>Birthday</option>
      <option value='midnight'>Midnight</option>
      <option value='other'>Other</option>
      <option value='reset'>Reset</option>
    </select>
  </div>
  </fieldset>
  <h2>Countdown Details</h2>
  <fieldset>
  <div class=row>
    <label>Update Every</label>
    <select id="select_interval" onChange="Countdown.updateInterval(this);" style="font-size:1.0em; text-align:left;">
      <option value='1000'>second</option>
      <option value='2000'>2 seconds</option>
      <option value='60000'>minute</option>
      <option value='3600000'>hour</option>
      <option value='86400000'>day</option>
    </select>
  </div>
  <div class=row>
    <label>Date Format</label>
    <select id="select_date_format" onChange="Countdown.updateDateFormat(this);" style="font-size:1.0em; text-align:left;">
      <option value='mm/dd/yyyy'>MM/DD/YYYY</option>
      <option value='yyyy/mm/dd'>YYYY/MM/DD</option>
      <option value='dd/mm/yyyy'>DD/MM/YYYY</option>
      <option value='mm-dd-yyyy'>MM-DD-YYYY</option>
      <option value='yyyy-mm-dd'>YYYY-MM-DD</option>
      <option value='dd-mm-yyyy'>DD-MM-YYYY</option>
    </select>
  </div>
  <div class=row>
    <label>Number Padding</label>
    <select id="select_padding" onChange="Countdown.updatePadding(this);" style="font-size:1.0em; text-align:left;">
      <option value=' '>none</option>
      <option value='0'>0</option>
      <option value='-'>-</option>
      <option value='♥'>♥</option>
      <option value='♦'>♦</option>
    </select>
  </div>
  <div id="spinning_wheel_date_container" class="row">
    <label>Event Date: <span id="sw_date_picked"></span></label>
    <a class="button blueButton" href="#dummy" onClick="SpinningWheel_AI.openEventDate();">Change Date</a>
  </div>
  <div id="select_date_container" class="row">
  <label>Event Date:</label>
  <table style="float:right;margin-right:-27%;">
    <tr>
      <td align="left" class="setup_cell">
        <select id="select_month" onChange="Countdown.updateMonth(this);">
          <option value='-1'>Month</option>
          <option>1</option><option>2</option><option>3</option>
          <option>4</option><option>5</option><option>6</option>
          <option>7</option><option>8</option><option>9</option>
          <option>10</option><option>11</option><option>12</option>
        </select>
        <select id="select_day" onChange="Countdown.updateDay(this);">
          <option value='-1'>Day</option>
          <option>1</option><option>2</option><option>3</option>
          <option>4</option><option>5</option><option>6</option>
          <option>7</option><option>8</option><option>9</option>
          <option>10</option><option>11</option><option>12</option>
          <option>13</option><option>14</option><option>15</option>
          <option>16</option><option>17</option><option>18</option>
          <option>19</option><option>20</option><option>21</option>
          <option>22</option><option>23</option><option>24</option>
          <option>25</option><option>26</option><option>27</option>
          <option>28</option><option>29</option><option>30</option>
          <option>31</option>
        </select>
        <select id="select_year" onChange="Countdown.updateYear(this);">
          <option value='-1'>Year</option>
          <option>2009</option><option>2010</option><option>2011</option>
          <option>2012</option><option>2013</option><option>2014</option>
        </select>
      </td>
    </tr>
  </table>
  </div>
  </fieldset>
</div>

<a id="counter_done" class="button" onClick="Countdown.acknowledgeDone(this);" href="#dummy">Countdown is done!<br/>OK</a>
</body>
</html>

Listing 2: Countdown JavaScript Implementation (countdown.js)

/*
  Copyright (c) 2009 Technetra Corp
  Released under The MIT License
*/
var Countdown;
/* BEGIN SpinningWheel Application Interface */
var SpinningWheel_AI = {
  openEventDate: function() {
    var obj = document.getElementById('select_event');
    var event_title = obj.options[obj.selectedIndex].text;
    if (event_title == "Birthday" || event_title == "Other") {
  	var now = new Date();
  	var days = { };
  	var years = { };
  	var months = { 1: 'Jan', 2: 'Feb', 3: 'Mar', 4: 'Apr', 5: 'May', 6: 'Jun', 7: 'Jul', 8: 'Aug', 9: 'Sep', 10: 'Oct', 11: 'Nov', 12: 'Dec' };
 
  	for( var i = 1; i < 32; i += 1 ) {
  		days[i] = i;
  	}
 
  	for( i = now.getFullYear(); i < now.getFullYear()+4; i += 1 ) {
  		years[i] = i;
  	}
      SpinningWheel.addSlot(years, 'right', Countdown.get_selected_year());
      SpinningWheel.addSlot(months, '', Countdown.get_selected_month());
      SpinningWheel.addSlot(days, 'right', Countdown.get_selected_day());
      SpinningWheel.setCancelAction(SpinningWheel_AI.cancel);
      SpinningWheel.setDoneAction(Countdown.updateSWDate);
 
  	SpinningWheel.open();
    } else {
      alert("Please select 'Birthday' or 'Other' Event Type To Customize Date");
    }
  }, 
  cancel: function() {
    //document.getElementById('sw_date_picked').innerHTML = 'cancelled!';
  }
}
/* END SpinningWheel Application Interface */
 
/* Countdown Closure */
Countdown = function() {
  var $ = function(id) { return document.getElementById(id); }
  var $getInnerText = function(id) { var id = $(id); return (id.innerText? id.innerText : id.innerHTML) }
  var $setInnerText = function(id, val) { // hack for Firefox 3.5
    id = $(id);
    if (id.innerText)
      id.innerText = val;
    else
      id.innerHTML = val;
  }
  var MSSEC = 1000;
  var MSMIN = 60 * MSSEC;
  var MSHR = 60 * MSMIN;
  var MSDAY = 24 * MSHR;
  var gMy_interval_timer;
  var gSelected_day, gSelected_month, gSelected_year;
  var gTitle_input_field = "Midnight";
  var gEvent_type_text = "Midnight";
  var gEvent_type_value = "midnight";
  var gTarget_time;
  var gDone = false;
  var gTotal_yeardays = 365;
  var gStart_month, gStart_day, gStart_year;
  var gMark_month = -1, gMark_day = -1, gMark_year = -1;
  var gDate_format = "mm/dd/yyyy";
  var gPad = " ";
  var gUpdate_date_mark = false;
  var gUpdate_interval = 2000;
  var gUpdate_interval_text = "2 seconds"; 
  var gLocalStore = {};
  var gIs_iPhone = (navigator.userAgent.toLowerCase().indexOf('iphone')!=-1);
 
  window.onorientationchange = function() {
      switch(window.orientation) {
        case 0:
          document.body.setAttribute('orient', 'portrait');
          break;
        case 90:
        case -90:
          document.body.setAttribute('orient', 'landscape');
          break;
      }
      setTimeout('Countdown.countDown()', 0); 
      setTimeout(scrollTo, 100, 0, 1);
  }
 
  window.addEventListener("load", function() {
      initApp();
      window.onorientationchange();
      Countdown.countDown(); // set up initial timer values, fields and overlays
      gMy_interval_timer = window.setInterval('Countdown.countDown()', gUpdate_interval);
  }, false);
 
  var storeItem = function(item, val) {
    if (window.localStorage) {
      window.localStorage[item] = val;
    } else {
      gLocalStore[item] = val;
    }
  };
 
  var retrieveItem = function(item) {
    var val;
    if (window.localStorage) {
      val = window.localStorage[item];
    } else {
      val = gLocalStore[item];
    }
    return val;
  };
 
  var clearItems = function() {
    if (window.localStorage) {
      window.localStorage.clear();
    } else {
      gLocalStore = {};
    }
  };
 
  var actionButton = function(btn, action, dsply) {
    if (typeof btn == 'string') {
      if (btn != '') $(btn).style.display = dsply;
    } else {
      btn.style.display = dsply;
    }
    if (action != '') Countdown[action]();
  };
 
  var store_date = function(y, m, d) {
    storeItem('selected_year', y);
    storeItem('selected_month', m);
    storeItem('selected_day', d);
  };
 
  var store_update_interval = function(i, t) {
    storeItem('update_interval', i);
    storeItem('update_interval_text', t);
  };
 
  var format_date = function(date_fields) {
    var fmt = "";
    switch (gDate_format) {
      case "mm/dd/yyyy":
          fmt = [ date_fields.m, date_fields.d, date_fields.y ].join('/');
          break;
      case "yyyy/mm/dd":
          fmt = [ date_fields.y, date_fields.m, date_fields.d ].join('/');
          break;
      case "dd/mm/yyyy":
          fmt = [ date_fields.d, date_fields.m, date_fields.y ].join('/');
          break;
      case "mm-dd-yyyy":
          fmt = [ date_fields.m, date_fields.d, date_fields.y ].join('-');
          break;
      case "yyyy-mm-dd":
          fmt = [ date_fields.y, date_fields.m, date_fields.d ].join('-');
          break;
      case "dd-mm-yyyy":
          fmt = [ date_fields.d, date_fields.m, date_fields.y ].join('-');
          break;
    }
    return fmt;
  };
 
  var setOverlay = function(obj, tp, offset) {
    var mfactor, wfactor;
    offset = (offset < 0) ? 0 : offset;
    offset = (offset > 59) ? 59 : offset;
    if (window.orientation==0) {
      mfactor = 138/60;
      wfactor = 136;
      tp += 52;
      height = "58px";
    } else {
      mfactor = 260/60;
      wfactor = 256;
      height = "36px";
    }
    obj.style.top = tp+"px";
    obj.style.height = height;
    obj.style.width = (wfactor-(offset*mfactor))+"px";
    /* obj.innerHTML = obj.style.width; */
    /* obj.innerHTML = tp+"px"; */
  };
 
  var formatNum = function(num, len) {
      var d, padding;
      var fmt = "…";
      if (num < 0) return fmt;
      fmt = "" + num;
      d = len - fmt.length + 1; 
      padding = new Array(d).join(gPad);
      return padding + fmt;
  }; 
 
  var setFields = function(days, hrs, mins, secs) {
      if (gUpdate_interval_text.match(/minute/)) {
        secs = -1;
      } else if (gUpdate_interval_text.match(/hour/)) {
        secs = mins = -1;
      } else if (gUpdate_interval_text.match(/day/)) {
        secs = mins = hrs = -1;
      }
      setOverlay($("days_overlay"), $("days").offsetTop, Math.floor((days*60)/gTotal_yeardays));
      setOverlay($("hours_overlay"), $("hours").offsetTop, Math.floor((hrs*60)/24));
      setOverlay($("minutes_overlay"), $("minutes").offsetTop, mins);
      setOverlay($("seconds_overlay"), $("seconds").offsetTop, secs);
      $setInnerText("days", formatNum(days, 3));
      $setInnerText("hours", formatNum(hrs, 3));
      $setInnerText("minutes", formatNum(mins, 3));
      $setInnerText("seconds", formatNum(secs, 3));
      //alert("setFields, days="+days+", hrs="+hrs+", mins="+mins+", secs="+secs);
  };
 
  var setupSelectElementByValue = function(el, val) {
    var i, o;
    o = el.options;
    for (i=0; i < o.length; i++) {
      if (o[i].value == val) {
        el.selectedIndex = i;
      }  
    }
  };
 
  var setupSelectElementByText = function(el, val) {
    var i, o;
    o = el.options;
    for (i=0; i < o.length; i++) {
      if (o[i].text == val) {
        el.selectedIndex = i;
      }  
    }
  };
 
  var setupDate = function(etype) {
    var targ_date, start_date;
    var now = new Date();
    start_date = new Date(now);
    gStart_day = now.getDate();
    gStart_month = now.getMonth();
    gStart_year = now.getFullYear();
    var selected_hour = 1, selected_minute = 1, selected_second = 1;
    switch (etype) {
      case "midnight":
        now.setDate(now.getDate() + 1);
        gSelected_day = now.getDate();
        gSelected_month = now.getMonth() + 1;
        gSelected_year = now.getFullYear();
        store_date(gSelected_year, gSelected_month, gSelected_day);
        break;
      case "newyears":
        now.setFullYear(now.getFullYear() + 1);
        gSelected_day = 1;
        gSelected_month = 1;
        gSelected_year = now.getFullYear();
        store_date(gSelected_year, gSelected_month, gSelected_day);
        break;
    }
    targ_date = new Date(gSelected_year, gSelected_month-1, gSelected_day, selected_hour-1, selected_minute-1, selected_second-1);
    gTotal_yeardays = Math.floor((targ_date.getTime() - start_date.getTime())/(1000*60*60*24));
    $setInnerText('start_date', format_date({m:gStart_month+1, d:gStart_day, y:gStart_year}));
    $setInnerText('target_date', format_date({m:gSelected_month, d:gSelected_day, y:gSelected_year}));
    if (!gIs_iPhone) {
      setupSelectElementByText($('select_day'), gSelected_day);
      setupSelectElementByText($('select_month'), gSelected_month);
      setupSelectElementByText($('select_year'), gSelected_year);
    }
    return targ_date;
  };
 
  var initApp = function() {
    if (retrieveItem('pad')) gPad = retrieveItem('pad');
    if (retrieveItem('event_type_text')) gEvent_type_text = retrieveItem('event_type_text');
    if (retrieveItem('event_type_value')) gEvent_type_value = retrieveItem('event_type_value');
    if (retrieveItem('title_input_field')) gTitle_input_field = retrieveItem('title_input_field');
    if (retrieveItem('date_format')) gDate_format = retrieveItem('date_format');
    if (retrieveItem('selected_day')) gSelected_day = parseInt(retrieveItem('selected_day'));
    if (retrieveItem('selected_month')) gSelected_month = parseInt(retrieveItem('selected_month'));
    if (retrieveItem('selected_year')) gSelected_year = parseInt(retrieveItem('selected_year'));
    if (retrieveItem('update_interval')) gUpdate_interval = parseInt(retrieveItem('update_interval'));
    if (retrieveItem('update_interval_text')) gUpdate_interval_text = retrieveItem('update_interval_text');
    document.getElementsByTagName('body')[0].setAttribute('device', gIs_iPhone?'iphone':'unknown');
    $setInnerText('update_interval_text', gUpdate_interval_text);
    $setInnerText('pageTitle', gTitle_input_field);
    $('title_input_field').value = gTitle_input_field;
    gTarget_time = setupDate(gEvent_type_value).getTime();
  };
 
  var setEventTypeResources = function(ett, etv) {
    storeItem('event_type_text', ett);
    storeItem('event_type_value', etv);
    gEvent_type_text = ett;
    gEvent_type_value = etv;
  };
 
  return {
    get_selected_year: function() {
      return gSelected_year;
    },
 
    get_selected_month: function() {
      return gSelected_month;
    },
 
    get_selected_day: function() {
      return gSelected_day;
    },
 
    doSetup: function() {
        var i, e;
        if (gMy_interval_timer != null) {
          window.clearInterval(gMy_interval_timer);
          gMy_interval_timer = null;
        }
        if (gSelected_day != -1 && gSelected_month != -1 && gSelected_year != -1) {
          if (gIs_iPhone) $setInnerText('sw_date_picked', format_date({m:gSelected_month, d:gSelected_day, y:gSelected_year}));
        }
        setupSelectElementByValue($('select_event'), gEvent_type_value);
        setupSelectElementByValue($('select_interval'), gUpdate_interval);
        setupSelectElementByValue($('select_date_format'), gDate_format);
        setupSelectElementByValue($('select_padding'), gPad);
        if (!gIs_iPhone) {
          setupSelectElementByText($('select_day'), gSelected_day);
          setupSelectElementByText($('select_month'), gSelected_month);
          setupSelectElementByText($('select_year'), gSelected_year);
        }
        if ($('title_input_field').value == "") Countdown.setTitleFieldResources(gEvent_type_text);
        if (gEvent_type_value == "other" || gEvent_type_value == "birthday") {
          $('title_input_field').disabled = false;
        } else if (gEvent_type_value == "midnight") {
          $('title_input_field').disabled = true;
        } else {
          $('title_input_field').disabled = true;
        }
    },
 
    doMain: function() {
        $setInnerText('pageTitle', $('title_input_field').value);
        var etv = $('select_event').options[$('select_event').selectedIndex].value;
        gTarget_time = setupDate(etv).getTime();
        gUpdate_date_mark = true;
        window.scrollTo(0, 1);
        if (gMy_interval_timer != null) {
          window.clearInterval(gMy_interval_timer);
        }
        gMy_interval_timer = window.setInterval('Countdown.countDown()', gUpdate_interval);
        window.setTimeout('Countdown.countDown()', 0); 
    },
 
    countDown: function() {
      var now, ms, diff, daysLeft, hrsLeft, minsLeft, secsLeft, today;
      now = new Date();
      now_day = now.getDate();
      now_month = now.getMonth();
      now_year = now.getFullYear();
      ms = now.getTime();
      diff = gTarget_time - ms;
      if (diff <= MSSEC && !gDone) { 
        daysLeft = hrsLeft = minsLeft = secsLeft = 0;
        if (gMy_interval_timer != null) {
          window.clearInterval(gMy_interval_timer);
          gMy_interval_timer = null;
        }
        $('counter_done').style.display = "block";
        $('settingsButton').style.display = "none";
        gDone = true;
      } else {
        daysLeft = Math.floor(diff / MSDAY);
        diff -= (daysLeft * MSDAY);
        hrsLeft = Math.floor(diff / MSHR);
        diff -= (hrsLeft * MSHR);
        minsLeft = Math.floor(diff / MSMIN);
        diff -= (minsLeft * MSMIN);
        secsLeft = Math.floor(diff / MSSEC);
        gDone = false;
      }
      setFields(daysLeft, hrsLeft, minsLeft, secsLeft);
      if (now_month != gMark_month || now_day != gMark_day || now_year != gMark_year || gUpdate_date_mark) {
        today = format_date({m:now_month+1, d:now_day, y:now_year});
        if ($getInnerText('today') != today) $setInnerText('today', today);
        gMark_month = now_month; gMark_day = now_day; gMark_year = now_year;
      }
    }, 
 
    setTitleFieldResources: function(et) {
      storeItem('title_input_field', et);
      gTitle_input_field = et;
      $('title_input_field').value = et;
    },
 
    updateEventType: function(obj) {
      var ett = obj.options[obj.selectedIndex].text;
      var etv = obj.options[obj.selectedIndex].value;
      setEventTypeResources(ett, etv); // sets gEvent_type and related fields
      if (etv == "other" || etv == "birthday") {
        Countdown.setTitleFieldResources(ett); // sets title_input_field and related fields
        $('title_input_field').disabled = false;
      } else if (ett == "Midnight" || ett == "New Years") {
        Countdown.setTitleFieldResources(ett);
        $('title_input_field').disabled = true;
      } else if (etv == "reset") {
        ett = "Midnight";
        etv = "midnight";
        alert("Clearing local storage & setting up default event 'Midnight'. Please reload Countdown app to see changes.");
        clearItems();
        Countdown.setTitleFieldResources(ett);
        setupSelectElementByValue($('select_event'), etv);
        setupSelectElementByValue($('select_interval'), "2000");
        setupSelectElementByValue($('select_date_format'), "mm/dd/yyyy");
        setupSelectElementByValue($('select_padding'), " ");
        Countdown.updateInterval($('select_interval'));
        Countdown.updateDateFormat($('select_date_format'));
        Countdown.updatePadding($('select_padding'));
        $('title_input_field').disabled = true;
     }
     gTarget_time = setupDate(etv).getTime();
     if (gIs_iPhone) $setInnerText('sw_date_picked', format_date({m:gSelected_month, d:gSelected_day, y:gSelected_year}));
     window.scrollTo(0, 1);
    }, 
 
    updateInterval: function(obj) {
      gUpdate_interval = parseInt(obj.options[obj.selectedIndex].value);
      $setInnerText('update_interval_text', obj.options[obj.selectedIndex].text);
      gUpdate_interval_text = obj.options[obj.selectedIndex].text;
      store_update_interval(gUpdate_interval, gUpdate_interval_text);
      window.clearInterval(gMy_interval_timer);
      gMy_interval_timer = window.setInterval('Countdown.countDown()', gUpdate_interval);
      window.scrollTo(0, 1);
    }, 
 
    updateSWDate: function() {
      var results = SpinningWheel.getSelectedValues(); 
      gSelected_year = results.keys[0];
      gSelected_month = results.keys[1];
      gSelected_day = results.keys[2];
      store_date(gSelected_year, gSelected_month, gSelected_day);
      $setInnerText('sw_date_picked', format_date({m:gSelected_month, d:gSelected_day, y:gSelected_year}));
      window.scrollTo(0, 1);
    },
 
    updateDateFormat: function(obj) {
      gDate_format = obj.options[obj.selectedIndex].value;
      storeItem('date_format', gDate_format);
      if (gIs_iPhone) $setInnerText('sw_date_picked', format_date({m:gSelected_month, d:gSelected_day, y:gSelected_year}));
      window.scrollTo(0, 1);
    },
 
    updateDay: function(obj) {
      gSelected_day = obj.options[obj.selectedIndex].text;
      storeItem('selected_day', gSelected_day);
    },
 
    updateMonth: function(obj) {
      gSelected_month = obj.options[obj.selectedIndex].text;
      storeItem('selected_month', gSelected_month);
    },
 
    updateYear: function(obj) {
      gSelected_year = obj.options[obj.selectedIndex].text;
      storeItem('selected_year', gSelected_year);
    },
 
    updatePadding: function(obj) {
      gPad = obj.options[obj.selectedIndex].value;
      storeItem('pad', gPad);
      window.scrollTo(0, 1);
    },
 
    turnActionButtonOn: function(btn, action) {
      actionButton(btn, action, "inline");
    },
 
    turnActionButtonOff: function(btn, action) {
      actionButton(btn, action, "none");
    },
 
    acknowledgeDone: function(obj) {
      obj.style.display='none';
      $('settingsButton').style.display='inline';
    }
  }
}();
/* END Countdown */

Listing 3: Countdown Stylesheet (countdown.css)

/*
  Copyright (c) 2009 Technetra Corp
  Released under The MIT License
*/
body {
  background: #cccccc scroll repeat 0 0;
  font-family:sans-serif;
  font-size:16px;
}
body[orient="portrait"] {
  height:460px;
}
body[orient="landscape"] {
  height:300px;
}
 
h1 {
  font-size:1.5em;
  font-weight:bold;
}
#main_subtitles > span > span:first-child {
  color:#356AA0;
}
body[orient="portrait"] #main_subtitles {
  font-size:0.56em;
}
body[orient="landscape"] #main_subtitles {
  font-size:0.60em;
}
body[orient="landscape"] #main_subtitles > span > br {
  display:none;
}
.divider {
  color:red;
}
body[orient="portrait"] #main_subtitles > span:first-child .divider {
  display:none;
}
 
#main_footer {
  position:absolute;
  margin-left:10px;
}
body[orient="portrait"] #main_footer {
  top:350px;
}
body[orient="landscape"] #main_footer {
  top:210px;
}
 
#update_interval_container {
  font-size:0.8em;
  float:left;
  margin-top:-4px;
  color:#356AA0;
}
table {
  border-spacing:0;
  width:100%;
}
body[orient="landscape"] table {
  width:50%;
  float:left;
  border:none solid gray;
}
 
body[orient="portrait"] .overlay { width: 136px; }
body[orient="landscape"] .overlay { width: 256px; }
 
#seconds_overlay {
  position:relative;
  left:0;
  background-color:#3F4C6B;
  -khtml-opacity:0.5; -moz-opacity:0.5;
}
#minutes_overlay {
  position:relative;
  left:0;
  background-color:#CDEB8B;
  -khtml-opacity:0.5; -moz-opacity:0.5;
}
#hours_overlay {
  position:relative;
  left:0;
  background-color:#FFFF88;
  -khtml-opacity:0.5; -moz-opacity:0.5;
}
#days_overlay {
  position:relative;
  left:0;
  background-color:#B02B2C;
  -khtml-opacity:0.5; -moz-opacity:0.5;
}
#counter_done {
  position:absolute;
  min-height:66px;
  width:50%;
  text-align:center;
  -khtml-opacity:0.8; -moz-opacity:0.8;
}
body[orient="portrait"] #counter_done {
  top:200px;
  left:80px;
  font-size:1.0em;
}
body[orient="landscape"] #counter_done {
  top:130px;
  left:115px;
  font-size:1.2em;
}
.ctr {text-align:center}
 
#countdown_box {
  width:100%;
  position:absolute;
}
body[orient="portrait"] #countdown_box {
  top:30px;
}
body[orient="landscape"] #countdown_box {
  top:40px;
}
.unit > div {
  position:absolute;
  text-align:right;
  font-weight:bold;
  padding-right:10px;
}
body[orient="portrait"] .unit > div {
  right:160px;
  height:58px;
}
body[orient="landscape"] .unit > div {
  right:204px;
  height:36px;
}
body[orient="portrait"] #days_box { top:60px; }
body[orient="portrait"] #hours_box { top:118px; }
body[orient="portrait"] #minutes_box { top:176px; }
body[orient="portrait"] #seconds_box { top:234px; }
body[orient="landscape"] #days_box { top:10px; }
body[orient="landscape"] #hours_box { top:46px; }
body[orient="landscape"] #minutes_box { top:82px; }
body[orient="landscape"] #seconds_box { top:118px; }
 
.unit_label {
  position:absolute;
  padding-left:10px;
  border-left:2px solid #ddd;
}
.unit span {
  padding-right:10px;
}
body[orient="portrait"] .unit span {
  font-size:2em;
  height:58px;
}
body[orient="landscape"] .unit span {
  font-size:1.5em;
  height:36px;
}
 
#days_box span[class="unit_label"] { color:#B02B2C; }
#seconds_box span[class="unit_label"] { color:#3F4C6B; }
#minutes_box span[class="unit_label"] { color:#73880A; }
#hours_box span[class="unit_label"] { color:#C79810; }
 
#sw-wrapper { display:block; }
body[orient="portrait"] #sw-frame { bottom:112px; }
body[orient="landscape"] #sw-frame { bottom:8px; }
 
/* alignment correction for Shiretoko */
.row > label {
  left:0;
}
div[class="row"] > input,select {
  margin-top:7px;
}
body[device="iphone"] #select_date_container { display:none; }
body[device="unknown"] #spinning_wheel_date_container { display:none; }

Fedora 11 Network Installation from GRUB

nilayan — Sun, 21 Jun 2009 17:24:10 +0000

Fedora 11 is finally out. Now I want to install it on a system running Ubuntu 9.04 and dual-boot between both systems. This can easily be accomplished by configuring the GRUB boot loader used by Ubuntu (and other popular Linux distributions, including Fedora). First I’ll setup a boot menu entry to launch the Fedora 11 installation program, Anaconda and then I will use Anaconda to perform a network installation of Fedora 11. Note I will be using Grub as an intermediate step in the installation of Fedora and also as the final boot loader for both distributions.

When I want to try out or install the latest release of a Linux distribution, I prefer to use a network installation for a couple of reasons. First of all, I’m lazy — I don’t have to download the installation ISO images and then burn them to CDs or DVDs. This is a big plus. Second, its very easy to setup GRUB for a network installation. Finally, I can be more eco-friendly by not throwing out old CDs/DVDs every time a new release comes out, which is quite often with Linux distributions.

Basically, a network install downloads the installation packages for the Linux distribution over a network. So you’ll need a relatively fast, reliable Internet connection (e.g., broadband cable modem, DSL). If you have a slow connection, you’re better off using the CD/DVD installation method.

First Steps

To get started, I boot into my Ubuntu 9.04 system. I log in as root in a Gnome Terminal window. First, let’s modify the GRUB boot loader menu by adding an entry to boot into the Fedora 11 installation program, Anaconda. To do this, open the GRUB configuration file /boot/grub/menu.lst in a text editor. Now, find the first OS definition. Look for the first line that starts with the word title, followed by a short title/description. On my system the first OS definition looks like

title     Ubuntu 9.04, kernel 2.6.28-11
root      (hd0,2)
kernel    /boot/vmlinuz-2.6.28-11 root=UUID=a128e38d-9a45-49c3-9863-43e1b30671ac ro quiet splash
initrd    /boot/initrd.img-2.6.28-11
quiet

Let’s add the following lines just above this first OS definition.

title     Fedora 11 Installation
root      (hd0,2)
kernel    /boot/fedora11-vmlinuz
initrd    /boot/fedora11-initrd.img

You’ll need to change the device name on the line highlighted in red, to match your system’s configuration. Run the following Bash script to find the correct device name on your system and to print the resulting GRUB root command.

bash -c 'A=abcdefghij D=$(mount|awk "/on \/ / {print \$1}") E=${D/*\/sd} DADDR=${A%${E:0:1}*} PART=$((${E:1}-1)) && echo "root (hd${#DADDR},$PART)"'

Next, we’ll download the Fedora 11 installation kernel (vmlinuz) and initial ramdisk (initrd.img) images from a Fedora mirror site. I’m using mirrors.kernel.org.

root@ubuntu:~# cd /boot
root@ubuntu:/boot# wget -q http://mirrors.kernel.org/fedora/releases/11/Fedora/i386/os/images/pxeboot/vmlinuz -O fedora11-vmlinuz
root@ubuntu:/boot# wget -q http://mirrors.kernel.org/fedora/releases/11/Fedora/i386/os/images/pxeboot/initrd.img -O fedora11-initrd.img

You should now have the two files fedora11-vmlinuz and fedora11-initrd.img in the /boot directory.

We’re ready to reboot the system. Remember to hit the ‘ESC’ key to see the GRUB boot loader menu during the initial system startup phase. Select the entry titled ‘Fedora 11 Installation‘ to launch the Fedora 11 installation program.

Once the installation program has started, it will step through common configuration tasks to prepare your system for a network installation. Pay special attention to the network connection (Configure TCP/IP) and Fedora installation repository (URL Setup) configuration tasks. If you’re unfamiliar with network installations, take some time to understand the process by visiting the following pages from the Fedora Installation documentation.

After completing the configuration steps, the actual installation process takes approximately 40 minutes.

Hands-on with Zmanda Recovery Manager 3.0 on Ubuntu Server

alolita — Sun, 03 May 2009 20:30:04 +0000

In this article we present a hands-on try-out of the most important features of Zmanda Recovery Manager 3.0 (ZRM) on Ubuntu. We ran a series of backups and restores using a MySQL database for a popular multi-user WordPress blog. The primary database table (wp_posts) contained approximately 500K rows, averaging 2Kb per row. In order to run backups of our WordPress database, we first set up a ZRM backup server, as well as created a MySQL user account on our MySQL server host. The hardware and software configuration we used is shown in Table 1.

**Table 1: Hardware / software configuration for ZRM 3.0 test drive**
	tahoe: ZRM Backup Server (w/ Local MySQL instance)	bigsky: ZRM Backup Client (Remote MySQL Instance)
Hostname	tahoe	bigsky
MySQL Server	Community Edition v5.0.51a	Community Edition v5.0.67
Operating System	Ubuntu Server 8.04.1	Ubuntu Server 8.10
Processor	Intel Core 2 Duo 3Ghz	Intel Atom N230 1.6Ghz
Memory	8Gb	2Gb
Storage	1Tb RAID 5	750Gb SATA

Installing Zmanda Recovery Manager

ZRM Enterprise edition provides two binary installation packages — ZRM server rapid installer (ZRM-enterprise-3.0-installer.bin) and ZRM client (mysql-zrm-enterprise-client_3.0-1_all.deb). First we will discuss the ZRM server installation, and then describe the ZRM client installation.

ZRM Server Installation

Figure 2: ZMC Rapid Installer

The rapid installer is a graphical interface which steps you through installation of the server software components: the ZRM server, Zmanda Management Console and its dependencies, and ZRM command-line tools. To launch the installer, we ran the following commands on our ZRM backup server host (tahoe).

root@tahoe:~# chmod +x ZRM-enterprise-3.0-installer.bin
root@tahoe:~# ./ZRM-enterprise-3.0-installer.bin

The installation process takes only a few minutes, during which the user must select how the Zmanda Management Console web server is accessed, whether by http or by https. The installation process ends with a dialog box prompting the user to launch the ZMC.

To complete the ZRM server installation, we downloaded the Zmanda License Key file (zmanda_license) and placed it in ‘/etc/zmanda/zmanda_license‘. We set file permissions to 644 (chmod 644 /etc/zmanda/zmanda_license) and changed file ownership to ‘root’ (chown root /etc/zmanda/zmanda_license).

ZRM Client Installation

The ZRM client for Ubuntu is a Debian package that you install using standard package management tools such as dpkg. The ZRM client should be installed if the MySQL server being backed up (ZRM backup client) is running on a separate machine from the ZRM backup server and you want to run raw backups. To install the ZRM client, we executed the following command on our MySQL server host (bigsky).

root@bigsky:~# dpkg -i mysql-zrm-enterprise-client_3.0-1_all.deb

Note that we configured the ‘mysql‘ system user on our MySQL server host (bigsky) with sudo privileges by editing the ‘/etc/sudoers‘ file and adding a line ‘mysql: ALL=(ALL) NOPASSWD: ALL‘. This grants the ‘mysql‘ user account permission to execute LVM system commands, necessary for Linux LVM-based filesystem snapshots.

Then we created a MySQL database user which ZRM used for backup and recovery on our MySQL server host (bigsky). Listing 2 shows the SQL statements we used to create the user account and grant privileges.

Listing 2: SQL statements to create user account and setup privileges

mysql> GRANT LOCK TABLES, SELECT, FILE, CREATE, DROP, INDEX, \
    -> SHUTDOWN, ALTER, INSERT, ALTER ROUTINE, CREATE ROUTINE, SUPER, RELOAD \
    -> ON *.* TO 'zrmbackup'@'tahoe.local.pri' IDENTIFIED BY 'somepassword';
Query OK, 0 rows affected (0.00 sec)

mysql> GRANT LOCK TABLES, SELECT, FILE, CREATE, DROP, INDEX, SHUTDOWN, \
    -> ALTER, INSERT, ALTER ROUTINE, CREATE ROUTINE, SUPER, RELOAD \
    -> ON *.* TO 'zrmbackup'@'bigsky.local.pri' IDENTIFIED BY 'somepassword';
Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)

We set up SSH for login without passwords by using public keys for the ‘mysql‘ system user on our MySQL server host (bigsky). This enabled the use of SSH connections, by the default SSH copy plugin, between the ZRM backup server (tahoe) and our MySQL server host (bigsky) for backup and restore tasks.

Doing backup and restore

The steps below are listed out in detail to serve as an example of how backups and restores can be performed with ZRM on Ubuntu.

Backup — Backup Level: Full, Backup Mode: Raw

Create backup set — We created a new backup set using the “sets” screen within the “Admin” tab. In the “Create backup set” panel, we selected a ZRM userid from the “Backup Set Owner” drop-down list (admin). We then entered a “Backup Set Name” to identify this backup set (wordpress_backup). Finally, we saved the backup set configuration by clicking the “Add” button.
Configure backup set — We clicked the “Backup” tab. Doing so gets us to the screen labeled “what”.
1. what — We filled in values for each panel — “MySQL Server Parameters”, “MySQL User Parameters”, and “What to backup”. Then clicked the “Save” button.
  - MySQL Server Parameters — select the “Server Type” (MySQL server), “Connection Type” (Port), “Port Number” (3306), “Host” (bigsky.local.pri), “MySQL Client Utilities Path” (/usr/bin)
  - MySQL User Parameters — “Username” (zrmbackup), “Password” (somepassword), “SSL Options” (leave blank)
  - What to backup — “Backup Source Type” (Specific Database(s)), “Select Database(s)” (mark checkbox next to wordpress_db).
2. where — In the “Backup Where Parameters” panel, we filled in values for “Destination Directory” (/var/lib/mysql-zrm), “Temporary Directory” (/tmp), and “Retention Policy” (2 weeks). Then clicked the “Save” button.
3. when — In the “Backup Schedule” panel, we filled in values for “Backup Level” (Full), “Time Range” (daily), and “Backup Time” (10Hr 15Mins). Then clicked the “Add Schedule” button to save these settings.
4. how — Although we did not modify the default values on this screen, you should double-check values for “Backup Mode”, “SSH User”, and “Email Address” (if you are using email notification).
Application configuration file backup — As part of this backup set, we wanted to preserve the WordPress configuration file ‘wp-config.php‘. ZRM requires an intermediate text file that lists the full path to each configuration file to be backed up, separated by newlines. Therefore, we created ‘/etc/mysql-zrm/wordpress_backup/app_conf_files_list‘, with the full path to wp-config.php on the first line. Next, we edited the ZRM backup set configuration file ‘/etc/mysql-zrm/wordpress_backup/mysql-zrm.conf‘ (on tahoe) and added the line ‘config-file-list="/etc/mysql-zrm/wordpress_backup/app_conf_files_list"‘. Note that you can also dynamically generate the list of application configuration files using a pre-backup plugin script available from the Zmanda network.
With the steps above we had now scheduled a daily full raw backup of all the tables in the ‘wordpress_db‘ database every morning at 10:15am.

Figure 3: ZMC Backup screen

Backup — Backup Level: Incremental, Backup Mode: Raw

Before running an incremental backup, we added some blog posts to our running WordPress instance, in effect modifying the underlying database.

Configure backup set — Using the “Backup Set” drop-down list at the top left corner of the ZMC, we selected “wordpress_backup”. Doing so displays the screen labeled “what” within the “Backup” tab. Next, we scheduled another backup job via the screen labeled “when”. In the “Backup Schedule” panel, we filled in values for “Backup Level” (Incremental), “Time Range” (daily), and “Backup Time” (12Hr 00Mins). Then we clicked the “Add Schedule” button.

Monitoring

As we ran on-demand backups, we were able to track their progress using the monitoring feature available via the “Monitor” tab of the ZMC. The “Backup Run Details” pane is updated live as each phase of the backup completes.

Restore

Identify recovery point — Using the “db events” (viewer) within the “Report” tab of the ZMC, we clicked on the backup timestamp (TimeStamp: 21:53:24) hyperlink to begin configuring the details of the restore process. Clicking the timestamp hyperlink displayed the screen labeled “what” within the “Restore” tab.
Configure restore settings
1. what — Information on this screen notified us that all databases would be restored from the last successful full backup. We clicked the “Next Step” button at the bottom to proceed further.
2. where — We reviewed the connectivity and user settings for the MySQL server to which the data would be restored. All the default values were left unchanged. We then clicked the “Next Step” button to proceed further.
3. how — The settings on this screen such as “Temporary Directory”, “MySQL Shutdown Options”, and “Copy Plugin Parameters” specify how your data is restored. The default values were left unchanged. We clicked the “Next step” button.
4. restore — Once we reviewed the restore settings on this screen, we clicked the “Restore” button to actually start the process.
In a few easy steps we were able to successfully restore the WordPress database from a full raw backup.

Figure 4: ZMC Restore screen

Reports

After running numerous backups and restores, the pre-defined reports we found most useful include the “Backup Status Report” and the “Application Impact Report”.

Application Impact Report — Provides backup job statistics such as “Time Taken”, “Read Locks Time”, and “Flush Logs time” — all of which let you tune your backup schedules to avoid database peak usage periods.

Backup Status Report — Gives you an at-a-glance status overview for every backup job that has been run. This can prove very useful for quickly tracking down error conditions across all your backup jobs.

In Summary

Our exercise to backup and restore a moderately sized WordPress MySQL database has demonstrated that Zmanda Recovery Manager 3.0 is a capable MySQL backup and recovery solution for the enterprise Ubuntu user. Whether you are a system administrator or a seasoned DBA, you can easily use ZRM’s friendly and functional GUI to schedule and run your MySQL data backups.

Zmanda Recovery Manager 3.0 for MySQL on Ubuntu Server

alolita — Fri, 01 May 2009 07:12:30 +0000

Responding to growing demand for a professional level backup and recovery solution, Zmanda, a leading vendor for open source backup solutions, has introduced an Ubuntu server version of their Zmanda Recovery Manager (ZRM) for MySQL.

At LinuxWorld 2008 in San Francisco, 451 Group analyst Jay Lyman observed that Ubuntu will continue to gain acceptance in the enterprise as organizations increasingly use free, community-driven Linux distributions and grow their own in-house Linux expertise. As a bellwether example, Wikipedia has switched its 400 servers to Ubuntu.

MySQL is equally gaining in importance to the enterprise. This same Wikipedia is also powered by MySQL. Acquired by Sun in late 2007, MySQL has been propelled further and further into the big leagues. As larger enterprises adopt Ubuntu together with MySQL, the need for a robust, well integrated database backup solution only grows.

To meet the widest possible range of backup solution requirements, Zmanda offers three versions of ZRM for MySQL on Ubuntu: the ZRM Enterprise edition, the ZRM Cluster edition and a ZRM Community edition. In this article, we’ll take a tour of the features in the latest release of ZRM Enterprise edition 3.0 for Ubuntu Server. We’ll focus on the product’s core capabilities for database backup and recovery and then we’ll look at some new features — application configuration backup, full-text search in ZRM’s management console (ZMC) and support for 64-bit platforms.

What is Zmanda Recovery Manager (ZRM)

ZRM Enterprise edition 3.0 brings a feature-complete backup and recovery solution for MySQL databases to Ubuntu Server. In this section, we look at the major components of ZRM and then survey the most important product features. We’ll also describe the support options available for ZRM.

ZRM Enterprise edition 3.0 includes the following three components: Zmanda Management Console, a command line interface and a plugin framework.

Figure 1: ZMC Login screen

Zmanda Management Console (ZMC) — ZMC is a web-based console that provides an easy to use GUI wizard for essential backup and recovery tasks. ZMC controls reporting and monitoring for all tasks in ZRM. An interface with Zmanda Network enables easy access to the latest product documentation and context-sensitive help.

Core functions such as “Backup”, “Monitor”, “Report”, “Admin”, and “Restore” are presented as a tabbed interface in ZMC. The Backup and Restore tabs break down the respective processes into individual screens. For example, in the Backup tab you’ll see links for each step of the backup process labeled “what”, “where”, “when”, “how”, “summary”, respectively. As the user navigates through each screen, they fill in the necessary information and proceed to schedule or run the backup. Similarly, in the Restore tab, links for “what”, “where”, “how”, and “restore” are displayed at the top. The user navigates each screen to complete the recovery process. The other tabs — “Monitor”, “Report”, “Admin” — follow the same pattern, with links for related tasks under each tab.

Context-sensitive help is available throughout the ZMC by clicking on the question mark icon at the top left corner of each screen. Also, common error messages are hyperlinked to the Zmanda Network knowledge base, where you can find detailed information about the error message and how to resolve it.

Command Line Interface (CLI) — The CLI provides an alternate scriptable interface to the major functionality of ZRM. Using the CLI tools, ZRM can be integrated with existing backup and recovery systems.

Plugin Framework — ZRM’s capabilities can be extended with plugins. Pre-backup and post-backup plugins execute custom tasks before or after backup jobs. Copy plugins are used to specify the data transfer mechanism (e.g., ssh, socket) between the ZRM backup server and remote MySQL server. Filesystem snaphot plugins support NetApp SnapManager, Solaris ZFS and Veritas VxFS. Also, storage volume snapshot plugins support Linux LVM, Windows VSS, and EMC CLARiiON SnapView. Note that plugins for NetApp SnapManager, Veritas VxFS and EMC CLARiiON SnapView must be purchased separately. Other plugins are available for dynamic rescheduling of backups and binary log parsing.

Core Features

ZRM’s core features are database backup and recovery, monitoring, and running pre-defined and custom reports.

Backup

ZRM is a flexible platform that lets you to backup multiple databases and tables from one or more MySQL servers.

ZRM offers a variety of options for scheduling backups. You can schedule backups during planned downtime or run instantaneous hot backups without taking your MySQL server offline. You can request daily, weekly, and monthly backups from the “Backup Schedule” panel under the “Backup / When” screen of the ZMC. It is also possible to schedule hourly backups by defining multiple daily jobs. Once defined, backup schedules can be modified as needed, for example, to avoid database peak usage periods by postponing a backup job using the pre-scheduler plugin.

ZRM handles multiple storage engines including MyISAM, InnoDB, NDB, Archive, and Falcon. Depending on the storage engine being used by your database table, ZRM automatically selects the appropriate backup method. If you are using the InnoDB hotbackup tool from Oracle, ZRM provides seamless integration with a management console for the tool. ZRM has been tested with MySQL versions 4.1.x and 5.x. To support legacy database installations, ZRM can also work with MySQL 4.0.

For large databases you may want to use ZRM’s raw backup feature. Raw or physical backups are exact copies of your data. This shortens the time needed for backups as well as for restores. Calculating storage requirements ahead of time is easier since raw backups are just a copy of your database files. However, the major drawback of raw backups is that they can only be restored to the identical MySQL server version and machine architecture used to produce the original backup.

ZRM takes advantage of filesystem and storage volume snapshots when performing raw backups. Snapshots capture changes to your data at a given point in time without having to copy the entire filesystem or volume. As noted in the Plugin Framework section above, ZRM supports filesystem snapshots for NetApp SnapManager, Solaris ZFS, Veritas VxFS, as well as storage volume snapshots for Linux LVM, Windows VSS, and EMC CLARiiON SnapView.

Logical backups should be considered if you need to support multiple machine architectures and your data must be highly portable. Logical backups offer table level granularity and can be performed without taking the MySQL server offline. However, logical backups are slower because MySQL server must fetch the database structure and content and then convert the data into a logical format, usually a text file with SQL statements. Due to the conversion, storage requirements for logical backups may be larger than the size of the original data source. To offset the conversion overhead, ZRM can be configured to compress, using gzip or other compression tools, the backup images produced.

ZRM supports full and incremental backups. Full backups save a complete copy of your database or table, whereas incremental backups save only the changes since the last full or incremental backup.

Many MySQL-backed applications depend on custom configuration files. ZRM can back up these files along with application data stored in the database for quick recovery of the entire application. Examples of such applications include Wordpress, SugarCRM, and MediaWiki. In ZRM 3.0, this feature is available by manually editing the ZRM backup set configuration file mysql-zrm.conf.

To protect sensitive data, ZRM can perform secure backups of remote MySQL servers by using standard SSL encryption over-the-wire as well as by using on-disk GPG encryption. ZRM ensures backup integrity with checksums of the backup image.

Recovery

ZRM provides continuous data protection (CDP) by integrating filesystem snapshots as well as by tracking MySQL database binary logs. CDP backup captures all changes to your data to enable fast, point-in-time data recovery. ZMC performs point-in-time recovery using selective incremental restores derived from MySQL binary log positions. Alternatively, the user can specify a recovery point (e.g., 2008-12-23 11:15:32) via the “Restore” tab of the ZMC.

ZRM offers a fine granularity of control over data recovery ranging from the entire MySQL server, database, table, down to individual transactions.

The ZMC Visual Log Analyzer (DB events viewer) parses MySQL binary logs to drive an easy-to-use GUI for browsing the contents of incremental backups, identifying recovery points (e.g., specific transactions), and monitoring operational and security aspects of your database. Starting with ZRM 3.0, the keyword search feature in Visual Log Analyzer takes advantage of MySQL full-text searching to provide accurate results more quickly.

Reporting

ZRM offers a diverse range of reports to help you stay informed about the status of your backup and recovery jobs. Automated reports generated after each backup job summarize the status of the job with useful information such as backup type (e.g., logical or raw), backup level (e.g., full or incremental), performance statistics, and backup image location. If configured, ZRM will send backup reports via email in HTML or text format. Backup reports available via RSS can be used to show backup status within other applications. In addition to automated backup summary reports, multiple pre-defined reports provide commonly requested information about backup jobs. Each pre-defined report can be customized with over 30 backup and MySQL data fields to display information specific to your needs.

Product Support Options

Zmanda provides three levels of support (Basic, Standard, and Premium) for ZRM Enterprise edition customers. All support levels come with access to the Zmanda Network resources, security updates and feature upgrades, and with web-based technical support. Standard and Premium support levels offer telephone technical support with unlimited support incidents. Premium support adds 24×7 technical support and guarantees a 3-hour incident support response time. Annual support pricing is based on the number of MySQL server instances being handled by ZRM.

Conclusion

ZRM 3.0 for MySQL on Ubuntu brings a polished, easy-to-use and full-featured, professional backup and recovery solution to Ubuntu, today’s most popular Linux distribution.

The Web-based management console lets you schedule and control backup and recovery tasks from the comfort of your favorite browser. There’s also a command line if you need it. Plugins let you extend ZRM. All the major MySQL storage engines are supported. There are also high performance options available, like raw backups and file system snapshots. Other features let you back up your application’s configuration files along with the databases it uses.

More advanced recovery facilities include CDP and fine grained, transaction level data recovery which can be initiated by an easy-to-use GUI log analyzer and events viewer. Details about backup and recovery jobs are easily tracked through pre-defined and ad-hoc reports. Finally, ZRM is bundled with a variety of customer assistance options ranging from Web to e-mail to telephone support.

With the release of ZRM 3.0, Zmanda provides features and support that meet the increasing needs of the enterprise for professional backup and recovery of MySQL on Ubuntu.

Resources

Zmanda Recovery Manager (http://zmanda.com/backup-mysql.html)
ZRM Wiki (http://mysqlbackup.zmanda.com/)

Toward Cleaning Up Out-of-Date or Broken Examples In The GNU Bash Project

radkins — Mon, 27 Apr 2009 06:04:43 +0000

I had great fun putting together the tutorial Discovering Web Access Latencies Using Bash Co-Processing. However, speaking about examples that demonstrate the fabulous features of GNU Bash…

Bash is an important and even critical software project. Yet it’s a shame that the project does not always keep its examples and illustrations up to date or even accurate. The project should strive to showcase Bash’s latest features in an authoritative way. And it should prune any items that are no longer relevant or that don’t actually work. A case in point involves the coproc examples located in the bash-4.0/examples/functions directory. Now that there is support for rudimentary (i.e., single instance) coprocessing in Bash4, the old coproc examples should be updated with native implementations.

But wait, that’s not all. There are at least two problems with the illustration of coprocessing as currently shown in the GNU Bash project files. As seen in the coshell.README file, the synopsis for using the coproc.bash example function is:

Figure 1. Excerpt from bash-4.0/examples/functions/coshell.README (DOES NOT WORK)

...
Attached to the message you will find coprocess.bash and coshell.bash.
Here's a brief synopsis of use:

coprocess open telnet localhost
while coprocess read il ; do # ← PROBLEM 1
  echo "$il"
  case "$il" in
    *ogin:*)
      coprocess print 'user'
      ;;
    *ord:*)
      echo 'pass' |coprocess print --stdin
      ;;
    *$ *) # ← PROBLEM 2
      coprocess print 'exit'
      break
      ;;
  esac
done
coprocess close
...

The first problem with this example is that the Bash builtin read command, which coprocess read eventually calls, is by default in line mode which means that it normally won’t release characters for consumption until it receives a carriage return. Therefore, in this illustration, characters that are not released by a carriage return will not be seen by the body of the case statment and consequently the desired login and password responses will not be executed. A fix for this problem requires restructuring the program (see Figure 2 below).

The second problem with this illustration is in the case statement itself. The ‘*$ *’ pattern produces a syntax error because the space in the pattern needs to be escaped. This has been fixed in the version below.

Figure 2. A corrected version of the coprocessing excerpt from bash-4.0/examples/functions/coshell.README

#!/bin/bash
 
. ./coproc_orig.bash
 
coprocess open telnet localhost
# initial read mode: blocks until read sees ':'
# (responds to Login and Password prompts which
#  are normally followed by ': ', not carriage returns)
while coprocess read -d: il ; do
  echo "$il"
  case "$il" in
    *ogin*)
      coprocess print 'lizbennett'
      ;;
    *ord*)
      echo 'pride&prejudice' |coprocess print --stdin
      break
      ;;
  esac
done
coprocess print 'exit'
# next read mode: blocks until read sees carriage return
while coprocess read il ; do
  echo "$il"
  case "$il" in
    # just logs off at echo of first prompt
    *$\ exit)
      break
      ;;
  esac
done
coprocess close

Now that we’ve fixed the example of the old approach to coprocessing, let’s put together some code to demonstrate the new way. The following is an updated — and correct — illustration of coprocessing as supported natively in Bash4. This example uses the new coproc command. It also handles user name and password input. In this example coproc invokes telnet to remotely execute a command given as a command line argument. The program can also run a stream of commands provided on stdin. Enjoy!

Figure 3. Native Coprocessing Example Using Bash 4.0 `coproc` Command

#!/bin/bash4
# Edit above line to point to where your Bash 4 executable lives
# e.g., /bin/bash if you are running a distribution like Fedora 11
# that supports the latest version of the shell
#
# Examples:
#
#     Save this program as test_native_coproc.sh in the current directory,
#     then run chmod +x test_native_coproc.sh,
#     and then execute commands like the following:
#
#     ./test_native_coproc.sh date
#     echo -e "date\nuname -r\nwho" | ./test_native_coproc.sh -i -u darcy -p secret
#
#######################################################################################
#
# process command line arguments
OPTIND=1
while getopts "ip:u:" c; do
  case $c in
    i)	USE_STDIN=y ;;
    p)	MYPASSWORD=$OPTARG ;;
    u)	MYUSER=$OPTARG ;;
  esac
done
shift $(( $OPTIND - 1 ))
ARGS="$@"
MYUSER=${MYUSER:-lizbennett}
MYPASSWORD=${MYPASSWORD:-pride&prejudice}
 
# set up telnet coprocess, log in and then execute a single command
# or stream of commands if using stdin
coproc telnet localhost
while read -d: -u ${COPROC[0]}; do
  echo "$REPLY"
  case "$REPLY" in
    *ogin)
      echo "$MYUSER" >&${COPROC[1]}
      ;;
    *ord)
      echo "$MYPASSWORD" >&${COPROC[1]}
      break
      ;;
  esac
done
case $USE_STDIN in
  y)
    cat >&${COPROC[1]}
    ;;
  *)
    echo "$ARGS" >&${COPROC[1]}
    ;;
esac
echo "exit" >&${COPROC[1]}
while read -u ${COPROC[0]}; do
  echo "$REPLY"
done

Discovering Web Access Latencies Using Bash Co-Processing

radkins — Sun, 26 Apr 2009 14:03:03 +0000

This tutorial describes a Bash co-processing script latency.sh (Listing 1) that generates and sorts web access latencies for a user supplied list of hosts. It demonstrates a co-processing scheme that supports multiple, simultaneous co-processes and works in all versions of Bash.

It is an updated and parallelized version of an earlier script with a similar purpose, ffmirror.sh (see http://grulos.blogspot.com). This new script uses customized Bash co-processing support for parallelization. In addition, unlike the grulos script, this script does not require a Bash executable that has been compiled for special network redirection (that is, with compiled-in support for /dev/tcp, etc.). You can use either tcp redirection or wget by specifying the desired mode as an argument on the command line. This is good news for Debian and Ubuntu users because these distributions normally do not ship with net redirections enabled. Note that while the Bash net redirection mode is a bit faster than launching and using wget, the latter may be more robust across many iterations of running the script against large lists of both existing and non-existent hosts.

Co-processing can reduce the performance profile of the principal work path of the script from linear, or O(n), to nearly O(1). The principal work path in our case is setting up host connections. In our implementation, collecting the latency results is still linear, but with a negligible run-time coefficient.

latency.sh can be run with the default wget connection strategy:

$ ./latency.sh < myhostlist

or with the Bash network redirection strategy where available:

$ ./latency.sh tcp < myhostlist

Using the 20-entry sample host list at grulos.blogspot.com, the time to run this script on a normal desktop can be under 2 seconds, while the original code requires over 18 seconds. This makes it feasible to run such a command under programs like /usr/bin/watch to create dynamically updating displays.

Figure 1: Trial run (using the original sample host data from grulos.blogspot.com)

# time ./latency.sh tcp < new_hostlist.txt
04/26/2009 19:40:49 running tcp version
270ms www.google.com
540ms www.redhat.com
900ms www.kernel.org
940ms www.microsoft.com
940ms www.slackware.org
940ms www.w3.org
960ms www.ibm.com
990ms www.debian.org
1000ms www.wikipedia.org
1000ms www.gentoo.org
1050ms www.altavista.com
1070ms www.yahoo.com
1070ms www.dell.com
1090ms www.netbsd.org
1100ms www.freebsd.org
1220ms www.openbsd.org
1860ms www.linux.org
(not set) www.shakakalasfdksasdfasdf.com
(not set) www.oooooooos.com
(not set) www.tttttttttt.org

real	0m1.982s
user	0m0.156s
sys	0m0.160s

Note that hosts whose latencies cannot be computed, for whatever reason, are labelled as ‘(not set)’.

The Bash co-processing function presented in Listing 2 below for coprocess.bash is based on a similar example from the GNU Bash distribution. The difference is that this co-processing function implements a simple queuing facility to handle multiple simultaneous coprocesses. This is essential for us to be able to parallelize our web access application. It also uses parameterized named pipes (fifos) for communicating between coprocesses.

The script latency.sh works under both Bash 3.x and Bash 4.0. It should be noted that the recent native support for co-processes in Bash 4.0 is limited to a single co-process at a time and so could not meet our need for multiple simultaneous web connections.

Listing 1: `latency.sh`

#!/bin/bash
#
# latency.sh - Generate and sort web access latencies
#              for a list of hosts
#
# Copyright (c) 2009 Technetra Corp
#
#    This program is free software: you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation, either version 3 of the License, or
#    (at your option) any later version.
#
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.
#
#    You should have received a copy of the GNU General Public License
#    along with this program.  If not, see .
# 
# This is an updated and parallelized version
# of ffmirror.sh (http://grulos.blogspot.com)
#
# To run the default wget based version:
#    ./latency.sh < hostlist.txt
# or for the tcp (Bash net redirection) version:
#    ./latency.sh tcp < hostlist.txt
#
###################################################################################
 
. ./coprocess.bash
 
WHICH_BACKGROUNDER=${1:-wget}
echo "$(date '+%m/%d/%Y %H:%M:%S') running $WHICH_BACKGROUNDER version"
 
chk_time () {
  local t=$(proc/uptime)
  t=${t%% *}
  t=${t/./}
  eval $1=$t
}
 
wget_backgrounder () {
  local host=$1 a b
  [[ -z "$host" ]] && read host
  chk_time a
  wget --spider -T 10 -q -O /dev/null $host && chk_time b
  echo "$((b-a))|$host"
}
 
tcp_backgrounder () {
  local host=$1 a b
  [[ -z "$host" ]] && read host
  chk_time a
  exec 2>/dev/null 3<>/dev/tcp/$host/80 &&
    # send a GET request and read 1st line of response
    printf "GET / HTTP/1.1\r\nHost: $host\r\n\r\n">&3 &&
    read -u 3 -t 10 -a response &&  exec 3>&- && chk_time b
  echo "$((b-a))|$host"
} 
 
# spin off parallel connections (the 'map' in 'MapReduce')
while read hostname; do
  # provide hostname to wget or /dev/tcp as command-line argument
  coprocess open qindex ${WHICH_BACKGROUNDER}_backgrounder "$hostname"
  # alternatively, send hostname to wget or /dev/tcp via pipe:
  #   coprocess open qindex wget_backgrounder
  #   coprocess put $qindex "$hostname"
done
 
# collect and process results (the 'reduce' in 'MapReduce')
coprocess size len
for ((i=0;i<$len;i++)); do
  # read result from pipe
  coprocess get $i c
  coprocess close $i
  h=${c##*|}
  d=${c%%|*}
  if [[ d -ge 0 ]]; then
    # bucket sort variant
    e=$((d*100))
    until [ "${slist[e]}" == "" ]; do
      ((e++))
    done
    slist[e]="${d}0ms $h" 
  else
    elist[not_set++]="(not set) $h"
  fi
done
 
printf "%sn" "${slist[@]}"
[[ ${#elist[*]} -ge 0 ]] &&
  printf "%sn" "${elist[@]}"

Listing 2: `coprocess.bash`

# coprocess.bash
#
#    Copyright (c) 2009 Technetra Corp
#
#    This program is free software: you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation, either version 3 of the License, or
#    (at your option) any later version.
#
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.
#
#    You should have received a copy of the GNU General Public License
#    along with this program.  If not, see .
#
###################################################################################
 
declare -i FIFO_IN FIFO_OUT FIFO_START=60
declare -a COPROCESSQ
 
coprocess () {
  local CMD="$1" N="$2"
  shift 2
  [[ "$N" == [[:digit:]]* ]] && 
      FIFO_IN=$((N*2+FIFO_START)); FIFO_OUT=$((FIFO_IN+1))
 
  case "$CMD" in
 
    open) 
      local M=${#COPROCESSQ[*]}
      FIFO_IN=$((M*2+FIFO_START)); FIFO_OUT=$((FIFO_IN+1))
      local fifo="/var/tmp/coprocess.$FIFO_IN.$FIFO_OUT.$$.$RANDOM"
 
      mkfifo "$fifo.in" || return $?
      mkfifo "$fifo.out" || {
        ret=$?
        rm -f "$fifo.in"
        return $ret
      }
 
      ( trap "rm -f $fifo.in $fifo.out" 0; "$@" <$fifo.in >$fifo.out ; ) &
      COPROCESSQ[M]=$fifo
      eval "$N=$M"
      eval "exec $FIFO_IN>$fifo.in $FIFO_OUT<$fifo.out"
      return 0
      ;;
 
    close)
      eval "exec $FIFO_IN>&- $FIFO_OUT<&-"
      [ "$1" = "-SIGPIPE" ] && return 1
      return 0
      ;;
 
    get)
      local old_trap=$(trap -p SIGPIPE)
      trap "coprocess close $N -SIGPIPE" SIGPIPE
      builtin read "$@" <&$FIFO_OUT
      local ret=$?
      eval "$old_trap"
      return $ret
      ;;
 
    put)
      local old_trap=$(trap -p SIGPIPE)
      trap "coprocess close $N -SIGPIPE" SIGPIPE
      builtin echo "$@" >&$FIFO_IN
      local ret=$?
      eval "$old_trap"
      return $ret
      ;;
 
    size)
      local M=${#COPROCESSQ[*]}
      eval "$N=$M"
      ;;
 
  esac
}

How to Setup Ruby on Rails for Fedora 10 and 11

nilayan — Wed, 22 Apr 2009 08:15:53 +0000

In this how-to, we will step through the process of setting up a Ruby on Rails development environment on Fedora 10 and Fedora 11 Beta. To follow along, just cut and paste the commands in a terminal window.

The instructions in this how-to are based on a fresh installation of Fedora 10 (32-bit/64-bit) and Fedora 11 Beta (32-bit only). During the installation process, we selected the “Software Development” package which includes the tools (e.g., gcc, svn, git, make) and libraries needed for building and compiling software.

1. Development tools

First, let’s check to see if you have the basic set of development tools — gcc, make and git. Although these tools are installed as part of the “Software Development” package, you can check if they are on your system by running ‘which gcc make git‘ at the command line.

[root@fc10 ~]# which gcc make git
/usr/lib/ccache/gcc
/usr/bin/make
/usr/bin/git

Note: On Fedora 10 64-bit systems, the full path to gcc is /usr/bin/gcc.

If the tools are not on your system, then you can install them by running the following command.

[root@fc10 ~]# yum install gcc make git

2. Databases: MySQL, SQLite

A typical Rails application is backed by a relational database. We will install both MySQL (v5.0.77) and SQLite (v3.5.9) for our environment. Recent versions of Rails default to using SQLite. If set up on your system, however, MySQL is also easily used.

[root@fc10 ~]# yum install mysql-server mysql-libs mysql-devel
[root@fc10 ~]# yum install sqlite sqlite-devel

3. Ruby language

Next, we’ll install the core packages for Ruby (v1.8.6.287). We’ll also need the two packages ruby-mysql and ruby-sqlite3 for database access from Ruby programs.

[root@fc10 ~]# yum install ruby ruby-devel ruby-libs ruby-mode ruby-rdoc ruby-irb ruby-ri ruby-docs ruby-mysql ruby-sqlite3

4. RubyGems

RubyGems is the packaging system for Ruby applications and libraries. Each application or library is known as a gem. By default, gems are installed in ‘/usr/lib/ruby/gems‘ (’/usr/lib64/ruby/gems‘ on Fedora 10 64-bit systems). RubyGems lets you easily install and remove gems using the gem command. We’ll install the latest version of RubyGems (v1.3.2).

[root@fc10 ~]# wget -q http://rubyforge.org/frs/download.php/55066/rubygems-1.3.2.tgz
[root@fc10 ~]# tar xzf rubygems-1.3.2.tgz
[root@fc10 ~]# cd rubygems-1.3.2
[root@fc10 rubygems-1.3.2]# ruby setup.rb

5. Rails

Now that we have a sparkling RubyGems toolset, the first gem we are going to install is the Rails (v2.3.2) web application framework. An advantage of installing Rails using the gem command is that package dependencies are resolved automatically.

[root@fc10 ~]# gem install rails

6. Mongrel

Rails provides WEBrick, a simple HTTP server that can be used to test applications during development. However, WEBrick is not suited for production environments. Mongrel is a web server commonly used for Ruby web applications. A typical configuration consists of a Mongrel cluster (several Mongrel instances) behind a front-end Apache web server. We’ll install Mongrel 1.1.5.

[root@fc10 ~]# gem install mongrel mongrel_cluster

7. JSON

JSON (JavaScript Object Notation) is a lightweight data interchange format - a sort of “low-fat” alternative to XML. We’ll install the Ruby implementation of the JSON specification, json (v1.1.4). This allows the conversion of Ruby objects to JavaScript objects for use by clients running on the browser.

[root@fc10 ~]# gem install json

8. SQLite3 Ruby

The sqlite3-ruby gem provides Ruby bindings for SQLite3 databases, the default database system for recent Rails environments. We’ll install version 1.2.4.

[root@fc10 ~]# gem install sqlite3-ruby

9. Ruby MySQL

The mysql gem provides a client-side MySQL API for Ruby programs. Before we install the gem, we’ll need to create symbolic links in ‘/usr/lib‘ (’/usr/lib64‘ on Fedora 10 64-bit systems) to point to the MySQL client libraries installed in ‘/usr/lib/mysql‘ (’/usr/lib64/mysql‘ on Fedora 10 64-bit systems).

Fedora 10

[root@fc10 ~]# cd /usr/lib
[root@fc10 lib]# ln -s mysql/libmysqlclient.so
[root@fc10 lib]# ln -s mysql/libmysqlclient.so.15
[root@fc10 lib]# ln -s mysql/libmysqlclient_r.so
[root@fc10 lib]# ln -s mysql/libmysqlclient_r.so.15

Note: On Fedora 10 64-bit systems, the symbolic links should be created in ‘/usr/lib64‘.

Fedora 11 Beta

[root@fc11 ~]# cd /usr/lib
[root@fc11 lib]# ln -s mysql/libmysqlclient.so
[root@fc11 lib]# ln -s mysql/libmysqlclient.so.16
[root@fc11 lib]# ln -s mysql/libmysqlclient_r.so
[root@fc11 lib]# ln -s mysql/libmysqlclient_r.so.16

Or for the brave, just run this on either Fedora 10 or 11:

[root@fc10 ~]# cd /usr/lib
[root@fc10 lib]# for f in mysql/libmysql*.so mysql/libmysql*.so.1[56]; do ln -s $f; done

Note: On Fedora 10 64-bit systems, the symbolic links should be created in ‘/usr/lib64‘.

Now, we are ready to install the Ruby MySQL (v2.7) gem by running the following command.

[root@fc10 ~]# gem install mysql

10. Riding the Rails

Now it’s time to test your installation. A great way to do this is to follow the simple tutorial at ‘Getting Started with Rails‘. Build the blogging application described there twice: once for SQLite3 and then again for MySQL. If the application can be built without errors, your database environments are set up properly.

That’s it! Your shiny new environment is waiting for you to build your next Rails application on Fedora using SQLite and MySQL.

Tutorial: Comparing Yahoo! and Google Maps APIs Side-by-Side

radkins — Sun, 19 Apr 2009 06:45:18 +0000

This tutorial compares the APIs of Yahoo! and Google maps by implementing a simple side-by-side map overlay rendered on a single HTML page. In this tutorial we will describe the details of using each API in JavaScript. Along the way we’ll compare a few of the differences between the two APIs. This material is suitable to be incorporated as part of a computer science lab covering the AJAX interfaces to the Yahoo and Google Maps APIs.

Both the Yahoo! and Google Maps APIs are easy to work with. The packages are similar. Both started out being based on mapping technology provided by Navteq. Today Google uses Tom Tom’s Tele Atlas data, while Yahoo has continued to use Navteq. But both companies source geographic data from a variety of other partners as well. The Google Maps API seems to support a few more features, especially in navigation and map types. In addition, the Google maps generally offer greater detail and more up-to-date content, especially when rendering international regions.

In order to compare both packages side-by-side we initially wanted to run each Maps API together on a single page. Unfortunately, we could not load both APIs together due to namespace collisions. A workaround was to load each package into its own iframe.

With this approach we were able to develop a web page that shows an initial default geographic location, the Golden Gate Bridge, San Francisco, side-by-side in each of the Maps APIs. Each map is rendered with a sample of its package’s standard navigation controls. On each map, a marker is placed at the center when the map is initially rendered. On a MouseOver event, this marker will display a pop-up picture of the target location, which in our case is the Golden Gate Bridge. The pop-up image will also provide a link to its associated Wikipedia entry.

The user can specify a new address for each map and is also be able to input different latitude and longitude coordinates to render a new map location. In addition, a fresh marker is placed anywhere on the map that the user clicks.

Figure 1: Comparing Maps APIs Side-by-Side — Screenshot

Try this example

Starting out with Yahoo! and Google Maps APIs

An introduction to the AJAX Maps API for Yahoo! can be found at Yahoo! Maps Web Services - AJAX API Getting Started Guide. An introduction to the Maps API for Google can be found at Google Maps API.

You must obtain your own Yahoo! app ID and Google key in order to run the application presented in this tutorial on your own server.

Overview of Our Application

Our application resides in an HTML page that contains two iframe elements. The JavaScript application in each iframe can call functions in the other iframe. This feature is used to synchronize the two maps. Cross-frame communication is limited to using simple numeric and string values as arguments to the method calls; complex parameters such as objects cannot be supported.

Figure 2: Comparing Maps APIs Side-by-Side — Overview

Parent Page

IFRAME layout

The parent page that defines the iframe layout for our application, as depicted in Figure 1, is shown in the following listing:

Listing 1: Parent page with `iframe` layout


          "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <title>Side-by-Side Yahoo! and Google Maps</title>
  </head> 
  <body> 
    <div style="width: 844px; background-color: #ccc;"> 
      <br />
      <h2 style='text-align: center; color: white;'>
        Side-by-Side  Yahoo! and Google Map APIs
      </h2>
      <br />
    </div>
    <div style="width: 844px;">
      <iframe src="yahoo_map_frame.html" name="yahoo"
              style="border: none; width: 420px; height: 800px; position: absolute;">
      </iframe>
      <iframe src="google_map_frame.html" name="google"
              style="border: none; width: 420px; height: 800px; float: right;">
      </iframe>
    </div>
  </body>
</html>

Map Application Walkthrough

In this section we will explain various details about how our application is constructed and how it uses each Maps API package. Full listings for each API solution are presented at the end of the tutorial.

Loading the Maps API

Each iframe loads its own Maps API package. The following