in

Emerald Hand

Emerald Hand, Inc. community home page.

This Blog

  • Home
  • Contact
  • About
  • Directory of Computers/Tech Blogs

Syndication

News

I'm back to blogging

Worm in liquid maze

Design and development of information management tools.

July 2006 - Posts

  • Sider inside

    I wrote about Sider before, what it is and why I'm working on it. I also maintain a wiki with Sider documentation. You can find more details about the platform there. Today I want to discuss how it works.

    Sider describes and stores all information using XML. Each Sider document is an XML document. This use of XML technologies not only saves time and simplifies development, but also makes it easier to extend Sider. A lot of people are familiar with XML and there is a lot of documentation describing every aspect of the technology.

    Types

    Sider types use XML Schema (XSD) to describe information structure and validate documents. XSD does have some limitations, but for now it will suffice. Type inheritance works through schema inheritance. Each new type must provide a new schema, but it's possible to extend, restrict, substitute or combine schemas from other types.

    In addition to the schema, each type provides two more features: constructors and script objects. Constructor is an XSL transformation that creates a valid document of the current type. When user creates a new document through File->New type XSL constructor is used to create that document.

    Type script object is a JavaScript object to provide view with document operations. When view is loaded all types (related to the current document) are loaded as well. The view doesn't have a direct access to the document, but instead relies on the type objects to manage it. This is similar to MVC pattern and promotes separation of concerns.

    Views

    Sider views show the document and allow the user to interact with it. Each view is displayed using hosted web browser control (IE currently, but in the future we will add support for Mozilla) and interacts with the user, with the document (through type objects) and with Sider itself (through window.external).

    When a document is loaded it is transformed using XSLT provided by the view. Currently, all views generate HTML files, but it is possible to generate SVG, for example, to show graphics in the browser.

    This use of web technologies has certain limitations (web browser can be slow if a page is complex, it can leak memory and so on), but at the same time it promises the widest support of different ways to show data (especially right now, when interest in the web technologies is on the rise and we see a lot of different libraries appear, such as DojoToolkit). Another big advantage is that we will be able to add support for document editing online.

    In addition to HTML views can use other web technologies, like Flash, SVG, Java, ActiveX. It will even be possible to use not browse-based technologies, such as .NET window forms or C++.

    View components

    View components are used by the views and exist to promote code reuse. They are horizontal view plug-ins (any view can use any view component). Example of view component would be a JavaScript library (such as Dojo), spell-checking support, and so on.

    Conclusion

    Sider platform binds everything together. It finds and loads a document, finds the types that describe and validate it, and views to show it. Sider generates HTML document using view transformation and displays it to the user. When the user wants to save the document the platform extracts updated document from the view, validates and saves it.

    This is why Sider is unique. It binds all these standard technologies (XML, HTML, JavaScript, etc.) to provide a simple, uniform information management experience for both users and developers.

    Share this post: Email it! | bookmark it! | digg it! | reddit!
    Readability Stats: Word Count: 593; Sentence Count: 39; Grade Level: 9.3, more info...
    Posted Jul 31 2006, 10:25 AM by Ornus with no comments
    Filed under:
  • Small CS updates

    I added some of the CS Modules. Some of the modules are quite cool and add interesting features to the Community Server. Each post will now have some statistics and links to share it.

    A system of plug-ins is important in our modern day, and we intend to have support for a large array of them in Sider. It's flexibility will depend on them.

    Share this post: Email it! | bookmark it! | digg it! | reddit!
    Readability Stats: Word Count: 65; Sentence Count: 5; Grade Level: 6.9, more info...
    Posted Jul 26 2006, 12:37 PM by Ornus with 6 comment(s)
    Filed under:
  • Why Sider?

    In my earlier post I wrote about Sider, an information management platform. Read it if you want to learn more about what it is. Today I want to explain why I think it's important and why I'm spending time on it. Actually, I'm not working alone. I'm doing most of the stuff, writing code, documentation, etc. My partner, Igor, helps by providing feedback, helping me analyze problems and supplying new ideas.

    Igor thought of the Sider and I liked it, so I decided to dedicate my time to it. We both store a lot of information on the computer, notes, e-mails, contacts, project plans, etc. By the way, information is a collection of data chunks with relationships between those chunks. A text document or a table is data, but a collection of text documents organized in such a way that connection between each document is visible, is information.

    Unfortunately, current state of managing data on computers is rather weak compared to what is possible. Most programs can handle only a few documents types, and almost all of them are not very extendable. For example, Microsoft Word is focused at working with text, Excel mostly works with tabular data, OneNote stores notes in a tree-like structure, Outlook stores tasks and e-mail in lists a tree (but each task node cannot be a tree node).

    As a result I need to split information I want to store on the computer between different programs. My to-do list is in one program - ToDoList, but for recurring tasks that I need reminder on I use Outlook. To store my notes I use Wikipad. To write this blog post I use Word, but to post it - w.blogger. To work with tables (and I do every day, to track time I spend on different projects) I use Excel. For mind maps I might use FreeMind. To work with my information I need to have 5-6 programs to be running and constantly switch between them.

    In most cases these programs work separately and I can't reference Wikipad note in a ToDoList task (I can reference wiki file, but usually that's not enough). This need to split up between programs happens because each program was created to operate a single domain and solve only a few problems. In addition many of these programs are inflexible and it's hard to extend them if I want to add a missing feature.

    Even when I can create custom extensions for the application, I need to learn it's language and API (Wikipad uses python, Microsoft Office - VBA or .NET, ToDoList is in C++). Moreover, extensions supported are limited in scope. I might be able to add a new view or report, but adding support for a new document type is usually impossible. At best I can try to adopt existing data types, but this is complicated, error-prone and time-consuming.

    Sider started off as a more theoretical project, but evolved into an application to address all these problems. Sider stores information in a most-suitable, more intelligent, innter-connected way (a graph is stored as a graph, a tree as a tree, a table as a table, etc.) Working with any information on the computer should be easier and more natural, and that's what Sider is intended to do. Sider makes it easier to reuse information by referencinf it, importing and exporting, and so on.

    Sider is flexible and easy to extend. There's no need to try and force existing data type to be suitable for your needs. You can either extend existing type or create new one from scratch, or combine both methods.

    Another goal for Sider is to present data in the best way for the user. What is best will change depending on what the user wants to do so it is important to support a lot of different views.

    Sider is about helping people to use computers for what they were invented for, to work with data. Of course, it will not solve all problems. When released it will be quite limited compared to most of the existing programs. It probably will never be as good as Word at working with text, or as good as Excel at working with tables. However, it will offer a new and unique approach at managing information and helping people to solve different problems.

    In the next post I will talk more about Sider architecture and technologies used. Hopefully it will be an interesting post.

    Share this post: Email it! | bookmark it! | digg it! | reddit!
    Readability Stats: Word Count: 758; Sentence Count: 45; Grade Level: 8.5, more info...
    Posted Jul 26 2006, 12:22 PM by Ornus with 5 comment(s)
    Filed under:
  • Microsoft support example

    Just stumbbled on this fine example of Microsoft support documentation. In a document describing why onload event might not fire suggested fix is - "To resolve this problem, add an onunload event handler. This turns on the onload event handler so that it will fire (although how it does this is not known)."

    From http://support.microsoft.com/kb/811577/en-us.

    Share this post: Email it! | bookmark it! | digg it! | reddit!
    Readability Stats: Word Count: 61; Sentence Count: 6; Grade Level: 6.0, more info...
    Posted Jul 24 2006, 09:25 AM by Ornus with no comments
    Filed under:
  • Introducing Sider

    For a couple of years I have been working on a program, Sider (stands for Structured Information Document Editor). I probably should have written about it some time ago, but I think I wasn't confident enough to do it. I am now. It's a project I'm working on full time and I hope it can make many people's life easier (as well as provide some income for me. I need to eat too you know :) )

    Sider is a data management platform. It aims to store information in a data structure that fits it the best, and show it in a manner that fits the user.It provides the tools to describe the data (types), how to show it (views) and takes care of storing the actual information in the documents. Sider makes it easy to relate different data to each other through inheritance, referencing and composition. For example, one type can describe an abstract tree and another describes an HTML document. Both of the types can be combined into a 3rd type to create a tree with an HTML in each node. This could be used to store notes.

    This support for any combination of data types is Sider's main and unique benefit. It makes it possible to use a single program to store all information together, even if different parts of it have very different structure. There are a lot of potential benefits because of that. Searching, cataloging and managing all of the information becomes much easier.

    Document

    All data in Sider is stored in the documents. Each document usually consists of a number of smaller documents. In above example of a tree with HTML nodes the tree is a document and each node is a document. This is a natural way of thinking about any information. It consists of smaller parts.

    Such implementation with each document consisting of smaller number of documents promotes reuse and actually makes it easier to create complex combinations. The rules on how to describe data are simple, unique and easy to understand, but the can be combined to create a large number of complex combinations.

    Type

    Each document is of some type. The type describes the structure of the data in a document, how to create a new document and how it can be modified.

    A type is similar to a class and a document is an object. A type can consist of other types, just like a class can have fields of other classes. A type can also inherit from another type to add support for a new type or extend existing structure. This mechanism is similar to the OO inheritance but more flexibl . This makes it easier to create new types by combining and reusing existing code. In many instances it's possible to slightly modify the base type to achieve the desired result. Inheritance also ensures data structure can be maintained easily.

    There are abstract types to provide common structures (a tree, a list, a table, and so on) and atomic types describing smallest unit of the data (phone number, person name, and so on. Any information that is hard to logically subdivide into anything useful). In most cases the user will see documents of concrete type, consisting of the abstract and the atomic type combinations.

    View

    Type just describes the data, but doesn't show it. Views are used for that. Through a view user gets to see the document and interact with it. There can be any number of views for any type to present the document in any suitable way. A common example is a table of records. It can be shown just as a data grid, as a chart or a report.

    Conclusion

    All of that is possible with some of the existing programs, but Sider takes it to the next level. It's a foundation on top of which new data management applications can be built. It takes care of finding types and views, loading them and initializing. It provides inheritance for the types, persistance and shows the document with the correct view. It hides all common code that developers have to write over and over again (save, load, etc) and helps him focus on solving the problem at hand and adding important, unique functionality.

    Documents can reference each other (and inside of each other) so all the information is connected. A PIM module for example, contacts can link to personal notes, notes can reference plans, everything is connected.

    Sider supports a system of plug-ins to provide horizontal functionality, such as spell checking, for different types and views.

    In the next post I'll talk more about why I'm creating this platform and what technologies I use to make it work.

    Share this post: Email it! | bookmark it! | digg it! | reddit!
    Readability Stats: Word Count: 785; Sentence Count: 49; Grade Level: 9.2, more info...
    Posted Jul 22 2006, 04:43 PM by Ornus with 2 comment(s)
    Filed under:
  • Frames leak in IE

    For the last couple of days I've been working on fixing memory leaks in IE. I'm hosting IE WebBrowser control in a local application (Sider). A lot of different pages are loaded in the application's life-time and memory leak is a serious problem.

    I have been fighting this issue in IE for a long time. The best solution would be to force IE control to unload and reload, releasing all memory it's holding, but I'm not sure if this is even possible. I never managed to find a way to do it.

    Currently, IE control is always loaded and I have to address the problem at the web page level (and scripts involved). Most common source of the leaks is circular reference between JS and DOM. Often this is caused by anonymous closures and anonymous functions when working with events.

    JS libraries (such as Dojo Toolkit) often try to address this issue by acting as a mediator between developer and the browser. They provide a method to register an event handler and clean it up (by breaking circular references) when the page unloads. In many cases this alone is enough to eliminate leaks.

    It wasn't in my case. The leaks continued and I had to find a cause (and a solution). In the past I have spent days looking for the exact source of the problem. I knew it was IE-related, but I never could figure it out. I was cleaning everything up, but IE continued to leak.

    In this round of me vs. IE I found the cause in about an hour (thanks to Drip). The problem is with frames. For whatever reason I have decided to show several pages together using frames. I found out Dojo causes leaks when it's loaded in a frame. The same page that unloads correctly when shown on its own leads to problems when shown in a frame. I suspect this issue related to how IE unloads frames. I would have never suspected! What's interesting is that Dojo causes a leak even if there're no widgets. For my test I just include dojo.js file on an empty page and leak is already there (only in frames). Strange...

    To be fair, I have to say that JSForms has this problem too. Actually it leaks more often than Dojo does. Eliminating memory leaks in IE is not simple at all.

    I have added an issue to the Dojo trac (defect ticket). I really do hope the problem can be resolved. Until then I see no other option but not to use frames. Dojo layout manager makes it much easier.

    Share this post: Email it! | bookmark it! | digg it! | reddit!
    Readability Stats: Word Count: 441; Sentence Count: 38; Grade Level: 5.3, more info...
  • Switching to Dojo

    I decided to go ahead and switch to the Dojo Toolkit. It provides almost everything I need and the next release looks very promising. It took me about a day to do the switch, but now I'm happy it's done. I had a chance to go over the code (as I was converting) and improve it.

    This means I'm not going to improve JSForms further. I'm going to take svn down, but I'll keep the wiki and the website for some time. However, there will be no further updates.

    I'm not going to stop writing JavaScript and I do want to contribute to the Dojo what I can. I also think I'll continue to write about it in the future.

    Share this post: Email it! | bookmark it! | digg it! | reddit!
    Readability Stats: Word Count: 120; Sentence Count: 9; Grade Level: 5.0, more info...
Copyright © 2006-2007 EmeraldHand, Inc. All rights reserved.
Powered by Community Server (Non-Commercial Edition), by Telligent Systems