Explore MINets: Redantant Exploring Multimedia Information Networks
ABSTRACT:

The abundance of multimedia data on the Web presents both challenges (how to annotate, search and mine?) and opportunities (crawling the Web to create large structured multimedia networked knowledge bases which can be used to conduct inference effectively). The proposed research has the ambitious aim of building a unified STRUCTURED networked database representation called Multimedia Information Networks (MINets). In MINets, concept nodes and multimedia data nodes will be connected together by the ontological and cross-domain links, covering both content and context knowledge. Characterizing a dense graph of relations among all concepts will provide a strong graph theoretical framework to study information fusion and inference strategies. Information fusion procedure will occur as a natural consequence of having a complete linkage representation among all kinds of nodes in the network.  In particular, we aspire to construct and utilize the MINets for recognition and inference tasks based on the complementary cross-domain links between multi-modal contents.  Moreover, the resultant MINets representation will also enable cross-domain correspondence for effective information transfer and inferences which would be otherwise much more difficult using traditional single-domain ontology structures, and therefore can unleash much richer knowledge resources for many fields.


SYSTEM ARCHITECTURE:




PROBLEM DEFINITION:

The goal of this approach is to annotate the images with some manually defined concepts, using visual and contextual features for learning a latent space. Specifically, by feeding the latent vectors into existing classification models, it can be applied to multimedia annotation, which is one of the most important problems in multimedia retrieval. Furthermore, we show a more sophisticated algorithm which can directly incorporate the discriminant information in training example for multimedia annotation without using mapping as a prestep. It jointly explores the context and content information based on a latent structure in the semantic concept space.

EXISTING SYSTEM:

The general approach of learning latent semantic space has been extensively studied in the field of information retrieval. Popular techniques include Latent Semantic Indexing (LSI), Probabilistic Latent Semantic Indexing (PLSI), and Latent Dirichlet Allocation (LDA). These algorithms have also been applied to multimedia domain for problems such as indexing and retrieval. For example, learn latent feature vectors by LSI for natural scene images, and the learned features can be used effectively with general purpose SVM classifiers.

DISADVANTAGES OF EXISTING SYSTEM:
Some preliminary results have shown the effectiveness of these algorithms; however, all these methods suffer from the problem with sparse context links, which we solve with the use of content links.


PROPOSED SYSTEM:
In this paper, we show that a compact latent space can be discovered to summarize the semantic structure in MINs, which can be seamlessly applied in the state-of-the-art multimedia information retrieval systems. Specifically, this algorithm maps each MO into a latent feature vector that encodes the information in both context and content information. Based on these latent feature vectors, MOs can be effectively classified, indexed, and retrieved in a vector space by many mature off-the shelf vector-based multimedia retrieval methods, like clustering, Re-Ranking for multimedia retrieval.

ADVANTAGES OF PROPOSED SYSTEM:

Our approach is a “general purpose technique” which can be leveraged to improve the effectiveness of a wide variety of techniques.

MODULES:

ü Indexing and Training Module
ü Annotation Construction Module
ü Mapping of the multimedia objects
ü Exploring Context Links
ü Exploring Content Links
ü Ambiguity Module

MODULES DESCRIPTION:

Indexing and Training Module

First module of our project is Indexing and training module for the whole set of images. Indexing is done using an implementation of the Document Builder Interface. A simple approach is to use the Document Builder Factory, which creates Document Builder instances for all available features as well as popular combinations of features (e.g. all JPEG features or all avail-able features). In a content based image retrieval system, target images are sorted by feature similarities with respect to the query (CBIR). In this module, we index the images and training the images accordingly.

Annotation Construction Module

These are the virtual links which are created as a result of user feedback (e.g., tags), and may be represented as the linkages between the MOs and the contextual objects such as tags. In the real-world contextual links, the number of user tags attached to an MO is usually quite small. In some extreme cases, only a few or even no tag may be attached to an object, which often leads to sparse contextual links. In such cases, it is hard to derive meaningful latent features for MOs because the determination of the correlation structure in the latent space requires a sufficient number of such contextual objects to occur together. A reasonable solution to this problem is to exploit the content links between MOs. In this paper, we will show how the content links can effectively complement the sparse contextual links by incorporating acoustical and/or visual information to discover the underlying latent semantic space.

Mapping of the multimedia objects

In this paper, content links represent the content similarities between MOs, i.e., those visually and/or acoustically similar objects are assumed to have strong content links between them. Content links contain important knowledge complementary to that embedded in context links. However, to the best of our knowledge, the existing latent space methods, LSI, PLSI, and LDA, cannot seamlessly incorporate the content and context links in a unified framework. Some attempts have been made to jointly model content and context information to learn the latent space. They quantify the MOs into visual words, which are treated in a way similar to some COs by linking them to MOs. However, such approaches greatly increase the number of parameters in the latent space model, and make it more prone to quantization-induced noise and overfitting due to the sparse context links. In contrast, we will show that content and context links can be seamlessly modeled to learn the underlying latent space. The content information does not have to be quantified into some discrete elements such as visual words. Instead, the content link structure will be directly leveraged to discover latent features together with context links. Therefore, we propose an elegant mapping of MINs to the latent space which can support an emerging paradigm of multimedia retrieval which unifies the information in context and content links. In other words, the goal of this approach is to annotate the images with some manually defined concepts, using visual and contextual features for learning a latent space.

Exploring Context Links

With the development of Web 2.0 infrastructures, rich context links are often connected to MOs on the media-rich web sites such as Flickr, Youtube, and Facebook. In contrast to pure content information, these links provide extra semantic information to retrieve and index MOs in the Web environment. For a simple example, the images of “sea” and “sky” have similar color features which are difficult to distinguish by similarity in content feature space. However, by leveraging the user tags in their context links and mapping them into a new latent space by LSI, PLSI, and LDA, they can be distinguished with the semantics in their COs. Context-based Multimedia Retrieval (CxMR) approaches have been widely used in many practical multimedia search engines such as Google Images, which utilize the context links such as surrounding text and user tags. Although the information in the context links is useful in many cases, they are often sparse and noisy. In some cases, it can lead to questionable performance, when the context contains much more irrelevant information to the mining process. This is often evident from the Google Image results when the images do not match the corresponding search at all.


Exploring  Content Links

The CMR approach attempts to model high-level concepts from low-level concepts extracted from the MOs. In a typical multimedia retrieval system like QBIC and Virage, the query is formulated by some example MOs and/or textbased keywords. Then, the relevant MOs are retrieved based on their content features. The advantage of CMR is that it is an automatic retrieval approach. Once the concepts are modeled, no human labels are required to maintain it.

Ambiguity Module

In this module we propose our contribution of providing more accuracy to the proposed system by enhancing using ambiguity resolving problem. Ambiguity is, Middle vision is the stage in visual processing that combines all the basic features in the scene into distinct, recognizable object groups. This stage of vision comes before high-level vision (understanding the scene) and after early vision (determining the basic features of an image). When perceiving and recognizing images, mid-level vision comes into use when we need to classify the object we are seeing. Higher-level vision is used when the object classified must now be recognized as a specific member of its group. For example, through mid-level vision we perceive a face, then through high-level vision we recognize a face of a familiar person. Mid-level vision and high-level vision are crucial for understanding a reality that is filled with ambiguous perceptual inputs. Thus in this module we resolve the problem of ambiguity and enhance the accuracy and propose an efficient system.

HARDWARE REQUIREMENTS:

•         System                 : Pentium IV 2.4 GHz.
•         Hard Disk            : 40 GB.
•         Floppy Drive       : 1.44 Mb.
•         Monitor                : 15 VGA Colour.
•         Mouse                  : Logitech.
•         Ram                     : 512 Mb.

SOFTWARE REQUIREMENTS:

•         Operating system           : - Windows XP.
•         Coding Language :  ASP.NET, C#.Net.
•         Data Base             :  SQL Server 2005

REFERENCE:
Guo-Jun Qi, Charu Aggarwal, Qi Tian, Heng Ji, and Thomas S. Huang, “Exploring Context and Content Links in Social Media: A Latent Space Method”, IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 34, NO. 5, MAY 2012