Since the dawn of the Semantic Web much effort has been exercised to bring its power to the desktop. One of the most important parts of this aim is to leverage existing information sources and make them accessible to 'semantic' systems. This information is contained within various structures maintained by the operating system and a multitude of existing 'legacy' applications. These structures include files, messages, documents, pictures, calendar entries and contacts in addressbooks. Ludger van Elst coined the term `native structures' to describe them and 'native resources' for the pieces of information they contain.

A successful Semantic Desktop system cannot deny the existence of native structures. Dealing with them is an issue that needs to be tackled on every attempt to realize the Semantic Desktop vision. What follows is an account of the approach adopted within the NEPOMUK Social Semantic Desktop project, followed by the specification of the NEPOMUK Information Element Framework (NIE), vocabulary developed to express native structures in RDF.

Note that the abbreviation NIE may refer to the NIE Framework as a whole or to the NIE Ontology - the core part of the framework. In most cases the former meaning is used, unless explicitly stated otherwise.

The problem is a difficult one given the multitude of applications and data formats. Previous semantic desktop projects (e.g. Haystack or Gnowsis) had to develop their solutions. Some attempts at standardization have been made (Adobe XMP or Freedesktop.org XESAM) but a definite standard hasn't emerged yet.

Apart from large metadata description frameworks there exists a considerable number of smaller single-purpose ontologies aimed at specific types of resources (e.g. ICALor VCARD). A broad array of utilities has been developed for extracting RDF metadata from desktop sources (see RDFizers website, for an overview).

Various problems have been identified with those ontologies. They are expressed in many languages, often aren't as detailed as one would like, sometimes contain outright bugs. NEPOMUK Information Element Framework is an attempt to build upon that experience, to provide unified vocabulary to describe typical native resources that may be of interest to the user. These resources are intended to serve as raw data for other semantic applications. They can be browsed, searched, annotated and linked with each other. It is hoped that this framework will help to achieve the critical mass of raw RDF data that will ignite rapid development of applications which will bring actual value to the user and make everyday work easier and more productive.

In order to justify the design decisions that were made when creating NIE, a short introduction on the ontology architecture of NEPOMUK is necessary. It can be conceptually divided in two tiers.

The lower one is the data tier. It stores the native data extracted from various data sources on the desktop. These include filesystems, mailboxes, calendars, and addressbooks. They are usually maintained by external applications, that are not part of NEPOMUK by themselves. The user still uses Kontact or MS Outlook for contacts and events, OpenOffice or Word for writing documents etc. The upper ontology layer is composed of concepts and instances that make up the Personal Information Model. They represent more abstract entities.

Native resources are extracted, converted to RDF and made available for Semantic Web software agents. An approach is to store converted RDF in a repository, making it queryable. Another approach is to adapt datasources dynamically, using virtual RDF graphs but in general, extraction and storing RDF in a queryable index is the common approach, also used by the other desktop search engines.

Data represented in NIE has three roles. First, NIE data is intended to be generated by an extraction process. Second, RDF-based systems can create NIE structures natively, without building on existing applications. Third, data expressed in NIE can be imported back to native applications. The ontology is therefore a mediator between semantic and native applications.

These roles explain some of the design decisions outlined below. They are a direct result of the following assumptions about the NEPOMUK architecture:

The authors of this document believe that these assumptions would also hold for other semantic desktop systems.

Requirements stated above led to certain guidelines for the ontology design. The following sections contain a brief outline of them. This is by no means a sound and complete set of axioms that are always correct, but a set of factors that were taken into account. They are given in their perceived order of importance. Developers of ontologies that will extend NIE to describe new kinds of data are strongly encouraged to be consistent with them.

The core of the NEPOMUK Information Element Ontology and the entire NIE Framework revolves around the concepts of nie:DataObject and nie:InformationElement. They express the representation and content of a piece of data. Their specialized subclasses can be used to classify a wide array of desktop resources and express them in RDF.

nie:DataObject represent physical entities that contain data. They are extracted from filesystems, archives, contact lists, mailboxes etc. The representation itself is usually not interesting to the user. In order for a piece of data to be actually understood, it needs to be interpreted as an appropriate type of an InformationElement. Such an interpretation makes it possible to inspect the content of a nie:DataObject correctly, and possibly divide it into parts - new nie:DataObject which can have their own interpretations. This process is iterative and continues as long as needed.

All resources on the desktop are basically related to each other with two most fundamental types of relations: interpretation and containment.

nie:DataObject represents a native entity the user works with. The usage of the term 'native' is important. It means that a nie:DataObject can usually be directly mapped to a sequence of bytes. Examples include a file, a set of files or a part of a file. The granularity depends on the user. nie:DataObject is atomic in the sense, that in order to distinguish any more data objects within, it has to be interpreted first.

The detailed list of nie:DataObject subclasses is subject to extension. It is envisioned that future applications will define their own nie:DataObject subclasses in their own ontologies. The NEPOMUK Information Element Ontology defines only the most generic properties deemed applicable to all nie:DataObject. More specific information is to be expressed using vocabularies extending NIE.

nie:InformationElement is a piece of information stored within a data object. Content-specific properties are defined as properties of an nie:InformationElement. It is separate from the nie:DataObject in order to make the interpretation independent of the representation. Research has shown that this flexibility is necessary to create a framework flexible enough to accomodate for the complexity of data structures present on the desktop. For example let's consider a mailbox. It is usually stored in a file (this is expressed as a FileDataObject interpreted as a Mailbox). An IMAP mailbox though is stored on a remote server, but from the interpretation point of view it doesn't differ from a file mailbox (in NIE this is expressed as a RemoteHostAddress interpreted as a Mailbox). Another representation may be a software service available through some interprocess communication mechanism (as is the case of the Outlook mailbox available via a COM interface). In NIE this is expressed as a SoftwareService interpreted as a mailbox). We have three completely different representations (file, remote host, software service) with an identical interpretation (Mailbox).

For a more thorough example see figure below. We see an in-file mailbox, containing an email, which contains an attachment. Notice that entities appearing on the right side can also appear at different levels. Apart from the already mentioned mailbox, there are other possibilities. A partition may be interpreted as a filesystem, but also a file containing an image of a DVD can be interpreted as a filesystem. A nmo:MailboxDataObject can be interpreted as a nmo:Message, but an .eml file too. The same applies to nco:Contact. We may have a single .vcf file containing a single VCARD, which is interpreted as a nco:Contact, but an email nfo:Attachment, or a nfo:RemoteDataObject may have the same interpretation.

This approach gives a uniform overview of data regardless of how it's represented.

nie:DataObject instances don't just come out of thin air. They are usually extracted from some data source. These are represented by instances of a nie:DataSource class. Each instance represents a native application or system, that manages information. A nie:DataSource instance aggregates information required by the data extraction component to gain access to the source. In the case of a nfo:Filesystem, this may be a path, for a nmo:Mailbox this may be the hostname, login and password etc.

Various subclasses of a nie:DataSource are envisioned. They may include filesystems data sources, calendar data sources, website data sources etc. The exact choice of types and properties is considered specific to a particular rdf extraction application. Data extraction applications are encouraged to provide their own data source ontologies. Such an ontology should contain data source types supported by the application coupled with properties necessary to gain access to them.

It is important to notice that certain entities (such as a mailbox or a filesystem) may appear both as a DataSource and as an InformationElement. They should not be confused though. They are completely different classes with completely different meaning. DataSource is an entity for which an rdf extraction framework of choice has an adapter for. DataObjects are extracted from it, either automatically when they appear (with some kind of monitoring process) or manually at the wish of the user.

The choice of subclasses of a DataSource depends specifically on the capabilities of the chosen rdf extraction framework. Some frameworks prefer to use an integrated approach (like the 'personal information space' from Beagle) where the Desktop is treated as a whole, and no specific data sources are distinguished. Other (like Aperture), work with a specific set of data source types (e.g. FilesystemDataSource, IMAPDataSource, IcalDataSource etc.). In the Aperture example a Filesystem instance is extracted from a FilesystemDataSource, just as all files.

The provenance of a DataObject is represented by the nie:dataSource property. In many usage scenarios it is important to keep track of the provenance of each data object. This information is useful for a number of reasons.

Note that this approach itself allows for an arbitrary hierarchy of data objects. It makes the structure of the extracted data independent of the capabilities of a particular extraction framework. It can be used by all-encompassing semantic desktop systems, as well as smaller libraries specialized in a single type of data source. The former would probably begin with some high-level concept such as nfo:HardDiskPartition or a nfo:SoftwareItem interpreted as an nfo:OperatingSystem and build a containment tree to the very bottom. The latter could begin their work somewhere in the middle (e.g. an mbox file crawler) and proceed as deep as they like.

As said before, The NIE Ontology is intended to serve as a foundation for a broad framework. Extensions are to be added to accomodate for various types of DataObjects and InformationElements. Each extension is a separate ontology, housed within a separate namespace. It should accept the assumptions outlined in section Basic design decisions- be expressed in NRL and try not to extend beyond native desktop resources. A NIE extension can define its own types of Data entities (subclasses of nie:DataObject) and their interpretations (subclasses of nie:InformationElement) or simply augment vocabulary provided elsewhere. The developers are encouraged to reuse classes and properties already defined in NIE (as stated Integration of ontologies) or provide specializations for them (subclasses and subproperties).