KnowledgeUser

"One of the effects of living with electronic information is that we live habitually in a state of information overload. There’s always more than you can cope with.”
-Marshall McLuhan

Knowledge Is Power

Knowledge Is Power And Discovering It Is The Ultimate Global Weapon Of Mass Creation (WMC)

I’ve been researching how we deal with information overload for the past few years. I find it funny that we all talk about the problem but very few seem to be willing to tackle it and come up with a solution. Even fewer orginazations seem to be willing to apply solutions. During my travels there have been trials, tribulations, some failures, and some successes as I try to apply new technologies to deal with the problems created by old technologies and the exponential increase in data and supposed information. Or is that infomercials? I’m still at it and I have not yet gone mad! In fact I think there is some hope on the horizon given the pace at which new technologies are being introduced.

During this research what I have found most interesting is that the concept of “information overload” was recognized over a decade ago yet we still don’t seem to be solving the problem. More importantly, medical web sites are starting to recognize “information overload” as a source of stress and anxiety attacks, and are providing recommendations on how to cope!. Go ahead and prove me wrong. Do a search from the popular search engines on the following topics (and use quotes to get more specific results):

Information Triage

Information Overload

Knowledge Discovery

Knowledge Signature

If you get the same results that I did, which is always doubtful with popular search engines today given that they try to target advertising to you based on “inside baseball information”, what you will find is that many of the results are from as far back as 1996 and as recent as 2004. It seems to me that studies dealing with “information overload” stop in early 2000 when popular search engines became popular. But that’s the basis for another Blog article on conspiracy theory and the advent of “advertising compromise of search results”. But let’s not go there so early in my new Blog as I don’t want to make enemies this early in the process.

During my ongoing research I was delighted to come across the following study that was done under the European Union umbrella organization for the advancement in Knowledge Discovery. Their 2003 paper, yes that’s three years ago already, is a fantastic and very enlightening article on the need for knowledge discovery systems and their potential impact on society. Obviously, the EU wants to be ahead of the Knowledge Discovery curve and then apply its research benefits in this area to keeping the EU at the forefront of the economic benefits that will come with it. And good on them because this is a global issue with international economic consequences and countries had better start taking the initiative if they want to be on the next way of the electronic world paradigm.   

The document can be found at the following URL:

http://www.kdnet.org/kdnet/control/trends_in_kd_research

They really describe the much more challenging problem facing society if we don’t learn to deal with information overload and develop methods to deal with it at a human level.

Full Title: Knowledge Discovery Roadmap

Version: D2.1. Interim Version

Author(s): Prof. Dr. Lorenza Saitta (coordinator) + see contributors list

Workpackage: WP 2 Trends in Research

It’s a fantastic document and really hits all of the major issues that society is facing today with respect to “information overload”, “information triage”, and “knowledge discovery” to make the world a better place.

Despite the fact that it is three years old already I highly recommend that people that are interested in dealing with “information overload” and “knowledge discovery” download and read it because it is really a great roadmap of where we need to go in this field if we are to reduce stress and anxiety created in our society by “information overload” and also create new economic advantages through new information triage and knowledge discovery technologies.

Here is a particularly relevant excerpt of the document that hopefully will get your interest. In the interest of Information Triage, which we promote so much, here is a Knowledge View version of it! If after reviewing the Knowledge View of this 70 page document you feel inclined to read the whole document, then, we will have accomplished our goal in life.

http://www.cirilab.com/KDiscover/index.htm

To access the Entire Document, simply click on Entire Document within the Knowledge View interface.

===============================================

What is “Knowledge Discovery” ?

With the capabilities of currently available systems for collecting, storing and organizing data, virtually every event in human life, every economic transaction, and every public act could be tracked and memorized. There is no doubt that data (especially in large amounts) implicitly encode knowledge, at the very least the model of their own generating process.

However, letting the data «speak» can be an elusive and resource consuming task. In fact, the technologies for collecting, transmitting, and archiving information have far outstripped our capabilities for processing it into useful information. This has severely limited our ability to put this information to inferential use for generating insightful hypotheses and drawing persuasive conclusions. The dramatic evolution in the availability of data in various forms from various sources is thus creating many fundamental challenges in computing, communication, storage, and human computer interaction issues.

Knowledge Discovery was born in

Detroit

, at the Workshop on “Knowledge Discovery in Databases”, organized by Piatetsky-Shapiro at IJCAI 1989. Later on, it has been defined (Fayyad et al.2) as the “extraction of interesting (non-trivial, implicit, previously unknown and potentially useful) information (knowledge) from databases” or other information repositories. Again, in terms of Kuhn’s classification, Knowledge Discovery works by “discovery”, sieving large amounts of data, in search of surprising facts.

Motivation for Knowledge Discovery Research

Even though it is true that Knowledge Discovery research has both inherited and developed a great variety of algorithms for a number of tasks, new problems, generating new types of data, emerge continuously. These problems often require new approaches, which may challenge the current state of the art of the field.

The increase of the amount of available data requires that existing algorithms be scaled up and become quicker and quicker. Also, new computational resources are to be taken into account, like images, audio, and video data. Moreover, real-time data streaming offers new opportunities for Knowledge Discovery in order to on-line monitor processes.

Finally, distributed data repositories and “grid” computing ask for new technological solutions.

Objectives

With the changing perspectives of society and the types of the users of the knowledge, Knowledge Discovery has changing objectives as well. In order to move from a task centred (performance first) to a human-centred (understanding first) vision of information handling, automated programs for Knowledge Discovery and data analysis should become familiar, easy-to-use instruments of everyday life.

In order to achieve this objective, learning and discovery programs should become capable of:

* Learn from data with complex structure, such as text, images, videos, audio

information sources and software programs

* Adapt the representation language to describe a knowledge-intensive world

* Select the relevant pieces of information and discard the irrelevant ones

* Exploit available domain knowledge and produce human comprehensible results

* Scaling-up to handle quickly very large amount of data

* Learn from experience and be adaptive

Source:

http://www.kdnet.org/kdnet/control/trends_in_kd_research

Project Acronym: KDNet

Project Full Title: European Knowledge Discovery Network of Excellence

Contract No.: IST-2001-33086

Project URL: www.kdnet.org

References

2 U.M. Fayyad , G. Piatetsky-Shapiro, and P. Smyth (1996). From data mining to knowledge discovery . In U.M. Fayyad , G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy (Eds.), Adavances in Knowledge Discovery and Data Mining (pp. 1-34). AAAI/MIT Press.

Thematic Navigation of a Blog Site

Mining the Blog Data Vein 

I'm new to the Blog scene so I won't profess to know how long they have been around, what the subculture is about nor how it is shaping the way information is stored and communicated. What I have experienced already however is that there is a lot of valuable Blog commentary out there that is hidden deep down in the automated archive vaults of a Blog site. Getting to the vein of valuable Blog commentary is a real challenge when you first arrive at a site that has been around for some time but that you are just starting to follow.

Which category should I open first and was the Blog of the day associated with that category?

What archive year should I start in?

What if I’m not using the right key word in my search?

Self Expressive Information

I believe that information should express itself thematically as opposed to me putting a label on it. And then the system should allow me to navigate those themes in order to find the document or Blog commentary that I am interested in reading. And once I find what I’m looking for, then I should be able to apply information triage techniques to get the gist of it so that can quickly determine if I should invest more time at it.

I’ve created a thematic based Knowledge Map of my Blog site to help the reader locate information not based on what day I posted the Blog comments or what category I think it should be in but rather based on its own self expressive thematic content. You can access this at:

http://www.cirilab.com/KUKMAP/taxonomyFrameset.htm

The Thematic Knowledge Map is easy to use. The Document Navigator on the left allows you to drill down through the Blog content based on major, minor, and sub-minor themes. As you do so you will notice that the related Blog articles become more focused. The Document Navigator acts as a search front end which having to understand Boolean!

When you have narrowed the Blog articles to the point that you have one or two you believe contain the themes you are interested in, simple click on the document to see a Speed Read version of it. See my Blog article on Information Triage for more information on how this works. At this stage if I’m interested I can click on Entire Document and go directly to the Blog article.

Now for my little Blog of three articles content navigation is not a major challenge. However, for Blog sites that are years old and have thousands of articles with many more thousands of follow on comments, the Thematic Knowledge Map concept sure would be helpful.

What do you think?

Arnold

Villeneuve

www.cirilab.com

Thematic Mapping of Unstructured Text with MindManager

Thematic Mapping of Unstructured Text with MindManager

Mindjet's MindManager does a fantastic job extracting a mind map from a Microsoft Word document that has really good structural formatting. When an author creates a Microsoft Word document with good Heading structure the Import into MindManager is excellent and the resulting mind map is superbly navigable. However, ask two people to create a document on the same subject and you can be sure they will use different formatting.

Furthermore, Heading based Formatting is not necessarily representative of thematic content. That is to say that the documents Table Of Contents will not take you to what is of interest to you but rather to what the author believes is important, in a structural sense. Can you rely on the Table Of Contents to help you find what is important to you within the document? I believe there are better ways of finding what you are looking for within a document that the document's Heading structure, and better yet, a better way of using MindManager to locate it.

It is estimated that over 90% of the data / information in the world is unstructured. I'm guessing here, but I suspect that less than 5% of that unstructured information is in the form of Microsoft Word documents. Of that 5% one can imagine that less than 10% have good formating that is applicable to MindManager in order to create great mind maps. What I am trying to point out is that the creation of mind maps based on heading formats, while a reasonable concept, is only one approach and is not enough. Most of the unstructured information in the world is unformatted. And if it is formated in some form or another, you can be sure that the format is not consistent.

And what about Adobe PDF files, Corel WordPerfect documents, plain text stored in Oracle / Open Text / Document Management / Content Management / Customer Relations Management systems? Many of them have not structure or what structure they do have is not universal and not parsed by MindManager. 

I use Mindjet's MindManager every day and even teach people how to use the product. You could say that I am a MindManager fan. However, as a MindManager user I recognize that more needs to be done to make MindManager more relevant in the world of unstructured text given that there is a 90/10 ratio of unstructured text to structured text stored in the world. And a majority of the unstructured text is not formated to Microsoft Word with appropriate headings that lend well to mind mapping.

Is there another way to dynamicaly mind map unstructured text?

Thematic Mapping of Unstructured Text

One solution is to look at unstructured text based on the thematic content within it rather than the heading based formatting that may or may not be correct or even exist.

What if software could read a document and extract a thematic DNA signature from it?

What if a thematic structure could be expressed in a hierarchical fashion?

What if the extracted thematic structure could be used to navigate the document's contents in order to locate just the relevant informaiton you are interested in on a paragraph by paragraph level?

What if this thematic expression of the document could be represented in MindManager?

How does thematic expression of unstructured text work?

The company I work with specializes in information triage. www.cirilab.com The first phase of the information triage process is to analyze a document and determine the Major, Minor, and SubMinor themes within the document and build a hierarchical thematic DNA signature of the document. The hierarchical thematic DNA signature of the document is navigable in that you can locate information within the document by simply clicking on themes within the document's thematic signature. Here is an example of a Knowledge View of a document which contains the Thematic DNA Signature.

www.cirilab.com/KV/KnowledgeView.htm

But enough of the complicated stuff. How does this translate to MindManager?  Well, because our software creates a thematically hierarchical XML formatted view of the document, it can be migrated to MindManager and represented graphically in a mind map. The following document demonstrates how this is done.

http://www.cirilab.com/KV/CIRI_Creates_Dynamic_Thematic_MindMapsV1.pdf

The really interesting thing about this is that the Cirilab software creates a MindManager mind map based solely on the thematic content of the document and nothing else. The unstructured text document is allowed to express itself in a structured format and then is dynamically represented in a MindManager mind map.

Here is the original document:

www.cirilab.com/KV/Walk_Through_V4.pdf

Here is the Knowledge View of the same document:

http://www.cirilab.com/KV/KnowledgeView.htm

Here is the same document represented in a thematic based MindManager based mind map:

http://www.cirilab.com/KV/Walk_Through_V4.JPG

I don't know. You let me know what you think of thematic based mapping of unstructured text within MindManager.

Arnold Villeneuve

www.cirilab.com

 

   

The Art Of Information Triage

Too Much Information

Introduction

Did you ever want to be able to read faster so you can get through the many reports, emails, and documents you receive each day? Are you dealing with information overload? One of the major complaints from many information workers today is that they simply cannot keep up with the volume and frequency of the information that is bombarding them everyday.  Stop for a moment and think about how much time you are going to spend for the rest of your life reading documents, reports, newspapers, emails. How many documents have you read only to determine that the information was not required after all. Imagine if we could reduce that time by half? What would you do with the extra time now available to you? I know what I would do!

One solution is to learn how to speed read but this is easier said than done. Another solution is to use software that creates a readable version of the document that is easier to review, one that allows you to navigate the document and locate just the content that is relevant to you, paragraph by paragraph. 

What if there was software that could help you understand what is in a document, or even collection of documents, without having to read all of the document? The software should be designed to read a document for you and provide you with a simple and easy to read and navigable knowledge view version of the document. It should allow you to apply information triage techniques used by experienced researchers to help you immediately get the gist of a document’s contents. The software should take you well beyond getting a summary of the document by allowing you to navigate the document to locate themes that you are interested in. This is a very important feature because it allows you to zero in on the information that is important to you.          

The Challenge of Information Overload 

Many organizations produce extensive amounts of information, some of it purposefully (reports and marketing material) and some of it as a result of doing business (electronic messages). A major challenge we all face is trying to determine what information we should spend time on and what information is not worthy of our attention.

Information Triage is a Five Step Process

1.        Does the document contain themes that I am interested in?

• If Yes, then continue.

2.        Does the Synopsis of the document demonstrate that I should read further?

• A one page version of the document allows you to make this decision.

3.        Does the Detailed Summary of the document demonstrate that I should read further?

• An 80% smaller document allows you to make the decision to proceed based on thematic content.

4.        You navigate the document thematically to find only the paragraphs that contain the information that you are interested in 

5.        You decide to read the full document.

That’s information triage at work for you. Imagine if after looking at the synopsis of the document you decided that it did not contain information that was relevant to your job, life, family. Why would you want to read more of it?

What would Information Triage look like in action?

1. What ever it is, the solution must be simple and immeidate. Something like, "Right click" on any typical unstructured text file (Word, PDF, WordPerfect, WebPage) and select Speed Read.

2. Review the Thematic DNA Signature of the document. The themes in the document should give you a good idea of what is in the document. If the Major and Minor themes are not what your interested in, why would you want to read the document?

3. A one to one half page Synopsis (Speed Read) view of the document should be presented that you can review once you have determined that the document contains themes of interest. For a 100 page document, the Synopsis would be about 1 page and based on the major themes in the document. The Synopsis should be based on the document's original contents.

4. Based on a quick review of the Synopsis, a more detailed summary version of the document should be available (Detailed Summary) should be available. The Detailed Summary should be a more extensive summary view of the document (Power Read). For a 100 page document, the Detailed Summary should be about 10 to 20% of the original document size.

5. A form of Thematic Navigation should be available to assist in navigating the document. You should be able to use the Thematic DNA Signature of the document to navigate and locate just the information that is important to you. You should be able to drill down through the documents Major, Minor, and Subminor themes in order to locate just those paragraphs that contain what you are interested in and nothing else. This means narrowing our search for content that is important to us and removing that which is not without having to know a foreign language like Boolean!

6. As part of your Information Triage process, if you have now determined that you need to read the actual document you can simply access the original document (Entire Document) and it will launch with the appropriate application (Word, Adobe, Word Perfect, etc).

Conclusion 

The benefits of speed reading are obvious: process more information faster and raise your comprehension level at the same time. But not everyone is adept at speed reading and learning the process takes time, patience, and lots of practice. Speed Reading software should help you accomplish similar results without having to learn how to speed read.

Information triage is a technique that allows you to save time by focusing on just the information that is relevant and important to you. Right Click and you are Speed Reading. It really has to be that simple! 

KnowledgeUser

www.cirilab.com