Multimodal interaction systems combine visual information (involving images, text, sketches and so on) with voice, gestures and other modalities to provide flexible and powerful dialogue approaches, enabling users to choose one or more of the multiple interaction modalities. They break down the barriers in adopting mobile devices for value-added services and the use of integrated multiple input modes enables users to benefit from the natural approach used in human communication. This paper deals with the main features of multimodal interaction and systems, starting from the definition of visual language given in Bottoni et al. (1995) and extending it to multimodality. Modal/multimodal message, interpretation and materialisation functions and multimodal sentence are defined. This paper introduces and formally defines the different classes of cooperation between different modes, introducing the time relationships among the involved modalities and the relationships between chunks of information connected with these modalities.

Keywords: COMPUTING JOURNALS; Computing Science, Applications and Software; Internet and Web Services

Document Type: Research article

DOI: 10.1504/IJWGS.2007.012638

Affiliations: 1: Istituto di Ricerca sulla Popolazione e le Politiche Sociali, Consiglio Nazionale delle Ricerche, Via Nizza 128, 00198 Rome, Italy. 2: Istituto di Ricerca sulla Popolazione e le Politiche Sociali, Consiglio Nazionale delle Ricerche, Via Nizza 128, 00198 Rome, Italy

source : http://www.ingentaconnect.com/content/ind/ijwgs/2007/00000003/00000001/art00005

Traditional URLs are simple and easy to use, but they’re also quite limited. For one thing, a URL only points at a single, complete document. More granularity than that, such as linking to the third sentence of the seventeenth paragraph in a document, requires the author of the targeted document to manually insert named anchors at the targeted location. The author of the document doing the linking can’t do this unless he or she also has write access to the document being linked to. Even if the author doing the linking can insert named anchors into the targeted document, it’s almost always inconvenient.

It would be more useful to be able to link to a particular element or group of elements on a page without having to change the document you’re linking to. For example, given a large document such as the television listings of Chapters 4 and 5, you might want to link to only one station or one show. There are several parts to this problem. The first part is addressing the individual elements. This is the part that XPointers solve. XPointers enable you to target a given element by number, name, type, or relation to other elements in the document.

The second part of the problem is the protocol by which a browser asks a web server to send only part of a document rather than the whole thing. This is an area of active research. More work is needed. XPointers do little to solve this problem, except for providing a foundation on which such systems can build. For example, the best efforts to date are the so-called byte range extensions to HTTP available in HTTP 1.1. So far, these have not achieved widespread adoption, mostly because web authors aren’t comfortable specifying a byte range in a document. Furthermore, byte ranges are extremely fragile. Trivial edits to a document, even simple reformatting, can destroy byte range links. HTTP 1.1 does allow other range units besides raw bytes (for example, XML elements), but does not require web servers or browsers to support such units.

For the moment, therefore, an XPointer can be used as an index into a complete document, the whole of which is loaded and then positioned at the location identified by the XPointer, and even this is more than most browsers can handle. In the long term, extensions to XML, XLink, HTTP, and other protocols may allow more sophisticated uses of XPointers. For example, XInclude will let you quote a remote document by using an XPointer to tell browsers where to copy the quote in the original document, rather than retyping the text of the quote. You could include cross-references inside a document that automatically update themselves as the document is revised. These uses, however, will have to wait for the development of several next-generation technologies. For now, you must be content with precisely identifying the part of a document you want to jump to when following an XLink.

source : http://www.cafeconleche.org/books/bible3/chapters/ch18.html

Accessibility languages and techniques can only function so far as the user understands them. In addition to that, features can be maximized with awareness. “The virtual buffer is referred to as Virtual PC Cursor mode in JAWS. Virtual PC Cursor mode is enabled by default when viewing HTML documents in supported applications (such as Internet Explorer and Firefox with JAWS 7.0), and can be toggled on and off using the keystroke combination Insert + Z. In this mode, the user has access to HTML elements and their attributes, such as the th element for table headings.

When in Virtual PC Cursor mode, JAWS responds inconsistently to client-side scripting events. JAWS is able to respond to basic events in Virtual PC Cursor mode, such as click and keypress, and will refresh the Virtual PC Cursor snapshot to reflect any changes made to the content. The problem when using Ajax is that new content isn’t usually added directly in response to these events; instead, the content is usually added through the onreadystatechange event of the XMLHttpRequest object. Interestingly, JAWS 7.0 responds to the onreadystatechange event with Firefox, but not with Internet Explorer. The lack of response to the onreadystatechange event along with authors failing to focus on the parts of the document that has changed is essentially why most Ajax applications are reported to either not work at all, or behave inconsistently. For changes to be reported successfully to JAWS in Virtual PC Mode (the default mode and the mode that allows the user to interact with the content), the only reliable method of adding new content in Virtual PC Cursor mode is as a direct result of a click, keypress, or mouseover event, which obviously isn’t good enough for an Ajax solution.

From version 6, JAWS introduced a command to refresh the Virtual PC Cursor using the keystroke combination Insert + Esc. This means that a JAWS user can use this keystroke combination to gain access to content that has been added dynamically. The problem is letting the user know that the content has changed. A solution we considered was to add content that prompts the user to refresh the virtual buffer as a direct response to a click/keypress event, and replace that content when the onreadystatechange event is raised in the XMLHttpRequest object. The thinking behind this approach was that users of JAWS 6 and later could use the Refresh Virtual PC Cursor command, and users of earlier versions of JAWS and other screen readers that use a virtual buffer would be able to respond to this by toggling virtual mode on and off — for example, using the keyboard combination Insert + Z twice would toggle the Virtual PC Cursor mode on and off in JAWS, refreshing the virtual buffer.”

Read the full article from Juicy Studio

First, Rick Strahl gives an excellent example. He wants to demonstrate a problem he has with ASP .NET. Then he talks about it: “Notice that the style sheet I defined in the master content page as well as in the master page (#pagecontent) end up above the Standard.css themes file.

The same is true if you attempt to add any controls to the Page.Header controls collection - whatever you do there ends up BEFORE the themes style sheet as far as I can tell or if you add styles manually to the header section.

This means you effectively can’t override the styles defined in the themes file which is uhm problematic in many situations. This bit me today once again as I was overriding one of the styles in the themes css and found that I couldn’t override the class.

There’s a way to work around this but it’s not XHTML compliant - you can embed styles and style sheet references into the page’s content which at least guarantees that the styles are rendered after the theme’s CSS.

Loading a CSS stylesheet from a Control

Another related issue I ran into today as I was building a small wrapper control around the jQuery Calendar control is how do you effectively load a CSS style sheet from a custom control, without actually rendering the style sheet multiple times?”

Read the full article at Mr. Strahl’s web log

Another article by expert Rick Strahl. “ASP.NET 2.0 includes many new data binding features that can significantly simplify a variety of data binding scenarios. Unfortunately, one very common scenario-simple control data binding-has changed very little. While this is a common scenario in forms-based Web applications, it remains for the most part a manual and time-consuming process.

Simple control data binding refers to the process of binding a single value to a property of a control-for instance, binding form controls like textboxes, checkboxes, radio buttons, or selected values of list controls to individual data or object values. This could benefit significantly from a consistent approach to handling data binding deterministically, as well as managing error trapping and display consistently. In this article, I introduce a custom extender control that can be used to bind any individual value from data or objects to any control property. It can extend and provide designer support for any existing control on a Web Form.

In the process, I’ll demonstrate a flexible and intuitive approach that provides consistent behavior for data binding and unbinding, validation, error handling, and error display in forms-based Web interfaces. I’ll also dig into the details of how the control is built and describe some of the interesting benefits.

The wwDataBinder control addresses simple control data binding. In more concrete terms, the control manages bindings that express concepts such as “bind the value of the DataRow’s CompanyName field to the TextBox’s Text property” or “bind the Item.Entity.Pk property to the SelectedValue property of a DropDown list.” The control manages the process of binding the specified data values to the specified control property, as well as unbinding the value back into the underlying data item. This is true two-way data binding.”

Read the full article hosted by MSDN

“Using JavaScript on your pages? Style sheets too? When you put complex JavaScript code and style specifications into the HEAD section, you may end up with more lines of code there than actual content in the BODY section! Use external files to make your pages load faster, reduce coding errors, and increase search engine appeal.

A W3C Approved Solution

Many good CSS and HTML techniques seem to require developers to dodge the guidelines recommended by the World Wide Web Consortium (W3C). Not for fun, but because many browsers don’t completely support the W3C specifications. But the W3C guidelines endorse external CSS files!

Other external file advantages include:

• Smaller pages: Use external files to replace many lines of JavaScript or style definitions with a single line of code that calls the external file. Less code equals faster loading pages - and happier visitors.
• Easier maintenance: Easily include the same code or formatting information on every page. Need to change a font-family, color, or variable name? Just change it once in the external file and you see the change on every page.
• Search engine friendly: Search engine spiders like to see important content at the top of the page. Some even rank earlier content as more important. Pages with important content close to the top of the page are more spider-friendly.

”

Read the full article from Netmechanic and learn to control the head section of your web pages.

There was a question on Mark Birbeck’s mind. Should Xpath feature CSS? “But while some sort of convergence of XPath and CSS selectors may seem an obvious thought to many, the CSS ‘language’ continues to resist being brought up to date, and instead exists in a strange, murky world, of ‘quirkarounds’ and ’standards-proprietary’ syntax. (As people require the ability to address other parts of the source tree, new selection mechanisms have to be added, but in a way that doesn’t affect existing rules — resulting in ‘quirky workarounds’, and syntax that is not shared by any other standard.)

Before we look at whether XPath fits with CSS, it’s worth looking at what exactly it is that CSS does. It may seem obvious — surely it just provides styling information based on a set of rules? — but if you break it down a bit you’ll find that the general mechanism at play could be used in many other situations. To understand CSS we need to look at its processing model and its addressing model.

The CSS Processing Model

The CSS processing model [2] says that in effect all properties that are supported by the current media type should be added to each element in the DOM tree (see step 4, in the processing model). This means that if our document were to be rendered on a screen, each element in our DOM tree would gain a color property (amongst others), since color is one of the many properties available when the media type is screen.

Let’s assume that this property is actually implemented as an attribute on the element itself, and let’s also prefix it with a namespace for CSS, so that it doesn’t get mixed up with other color attributes that we might have. ”

Read the full article at the author’s personal blog.

Rick Strahl understands what he’s doing, and shares his expertise in this older article. He has some interesting suggestions: ”

What’s wrong with Data Binding in ASP. Net?

Unfortunately, I personally think that data binding in ASP.Net doesn’t go nearly far enough. First the process of assigning data sources is cumbersome at best using either a slow and work extensive designer or alternately by having to embed yet another script based markup tag (<%# %>) into source code. Both are way too cumbersome if you’re dealing with a lot of data on a regular basis.

But more importantly data binding in ASP.Net is one way only. You can only bind data to data but there’s no mechanism to unbind the data from the control back into its underlying datasource. It’s hard to really call ASP.Net’s mechanism data binding because really what it is is a data display mechanism.

To clarify though, there are really two types of data binding in ASP.Net. There’s list based data binding of the type that you use to bind data to a ListBox or a DataGrid. This mechanism actually works very well and provides a good amount of flexibility. This is also primarily a display mechanism – you tend to display data or lists with this type of binding.

Then there is simple Control Data binding which basically binds to a single value to a property of a control. A text box binding to a field of the database is an example of simple data binding. This also tends to be the most common data binding that most people do during data entry and the one that is the most time consuming. And here is where the problem lies – the control binding here is one way and involves some convoluted syntax that isn’t even property based.”

Read the full article hosted by West Wind Technologies.

“The type of a PNG image is defined in the IHDR image header. The image has a certain bit depth, up to 16 bits per sample, and a certain color type, from Grayscale to RGB+Alpha. If two PNG files of different types represent exactly the same image, each file can be regarded as a lossless transformation of the other. A lossless transformation can reduce the uncompressed stream, and such a transformation is named image reduction. In most cases, image reductions are capable of reducing the compressed stream (which is, in fact, our interest), as an indirect effect of reducing the size of the compressor’s input.

The possible image reductions are:

• Bit depth reduction
  The bit depth can be reduced to a minimum value that is acceptable for all samples. For example, if all sample values in a 16-bit image have the form (256+1)*n, (e.g. #0000, #2323, #FFFF), then the bit depth can be reduced to 8, and the new sample values will become n, (e.g. #00, #23, #FF).
• Color type reduction
  - If an RGB image has 256 distinct colors or less, it can be reencoded as a Palette image.
  - If an RGB or Palette image has only gray pixels, it can be reencoded as Grayscale.
  A color type reduction can also enable a bit depth reduction.
• Color palette reduction
  If the color palette contains redundant entries (i.e. duplicate entries that indicate the same RGB value) or sterile entries (i.e. entries that do not have a correspondent in the raw pixel data), these entries can be removed.
  A color palette reduction can also enable a bit depth reduction.
• Alpha channel reduction
  If all pixels in a Grayscale+Alpha or an RGB+Alpha image are fully opaque (i.e. all alpha components are equal to 2^bitdepth-1), or if the transparency information can be stored entirely in a (much cheaper) tRNS chunk, the alpha channel can be stripped.

There are, however, a few cases when some image type reductions do not necessarily lead to the reduction of the compressed stream. The PNG-Tech site contains experimental analyses of these possibilities; for example, see the article 8 bits per pixel in paletted images.

Interlacing, useful for a faster, progressive rendering, is another component of the PNG image type that affects compression. In an interlaced stream, the samples corresponding to neighboring pixels are stored far away, hence the data in it is less correlated and less compressible. Unlike JPEG, where interlacing may improve the compression slightly, the PNG interlacing degrades the compression significantly.”

Read the full article, learn more about .PNG Optimization.

While a little bit out dated for its testing realm, this article holds some still useful information for the modern developer. “XML in the browser has been the subject of many spirited discussions about bleeding-edge web development. Some feel that XML in place of HTML isn’t ready for prime time due to the lack of user agents that can properly parse and render it. Others feel that XML really belongs on the server or used solely as a descriptive framework for data and has no place in the visual world of the Web which is already adequately served by HTML.

Despite this, the newest generation of browsers possess powerful XML capabilities. The recent releases of Mozilla offer a parser and rendering engine that support XML technologies such as XML stylesheets, XML namespaces, XLink, SVG, and MathML. Along with its native support for SOAP and forthcoming implementations of WSDL and XSLT, Mozilla is poised to become a power player among XML client software.

Mozilla also permits XML to be rendered in the browser with CSS, and manipulated with the DOM. This is a real boon to those of us eager to experiment with XML transformations (both visual and structural) without having to delve into unfamiliar technologies such as XPath, the verbose traversal language of XSLT. If you’re already familiar with CSS and DOM, you’re more than halfway to achieving XML transformations in Mozilla.

This article demonstrates how to render XML in the browser with a minimum of CSS and JavaScript. The examples that follow were written for and tested with Mozilla 1.0 for Mac OS X and Windows 2000. This article is not a comprehensive tutorial on DOM, CSS, or XML, and a basic understanding of each of these technologies is required.”

Read the full article hosted by Apple’s own Developer Connection.