Monday, November 24, 2008

The Failed Psychological Contract with the American People

The Psychological Contract represents the emotional relationship between our leaders and ourselves, between bosses and employees, between parents and kids. The earliest manifestation occurs soon after birth and continues on throughout our lifetime.  For the Psychological Contract to succeed requires a continuous process of renegotiation. The current political establishment, on both sides of the aisle have failed us, putting ideology ahead of the greater well-being of the American people.

Today as we sink deeper into a fiscal and psychological depression, sameness is now different. We have been abandoned, left on the proverbial doorstep, to fend for ourselves, and unable to do so. We are lost in a familiar place.

The financial crisis is every bit a psychological crisis, in our institutions, our government, our leaders and ourselves. We have given up emotionally on those we rightfully expected to look after us. No wonder the markets seem to be in a free fall, after all, we are in a massive emotional free fall.

Our institutions look the same, they are not. The mall is still standing and vacant. The many vulnerable among us are in despair. Our current leaders, demonstrate a pathological narcissism, caught in the myth of their righteousness, are grotesque caricatures of what leadership ought to be.

And now, we hang our fragile emotional hopes on one man representing the good-enough parent to a population of his children. Hopes riding on a new set of leaders. He will fail and so will we if we don't hold one another accountable. We will fail if he and we do not successfully navigate the treacherous psychological waters of re-establishing a new trust in this fragile time. President Elect Obama has suggested that we rise and fall as one people. He is right. Let us hope we look beyond our projected expectations of one man and administration. The hope springs from us as a people, not bestowed on us by our leaders. Let us remember we are entering into a new psychological contract as peers and equals.

Let us take responsibility for our successes and challenges by setting clear expectations of what is possible, one building block of trust at a time.

Thursday, November 13, 2008

Federated Search is Good-the Semantics for True Contextual Search is the Tough part

I came across this recent post  The evolving federation of search’ http://www.kmworld.com/Articles/Editorial/Feature/The-evolving-federation-of-search--51415.aspx and thought the money quote from Sue Feldman was "If you understand you can get at this information without having to sort it or structure it in any way ahead of time—although you certainly can—and that it doesn't require that you have the same structure across multiple information sources—that's really the beauty of these technologies”

Yes federation of search produces better findability but that’s the easier issue to address.... its the semantics for true contextual search that’s the tough part.

The enterprise and web information access and search market is bursting at the seams with new products that focus on Semantic Enhancement, which is designed to add contextual meaning across a variety of enterprise applications, databases, content repositories and the Web.

Most of today’s solutions that index and normalize information (data and content) fail to completely resolve  metadata and contextual semantic relationships. More comprehensive methods that require building and maintaining ‘knowledge models’ are human capital intensive and expensive.Today, tens of millions of dollars are spent on paying skilled knowledge workers to "manage" metadata taxonomies and ontologies.

While ‘knowledge models’ and machine AI can approximate to the skill, accuracy and speed of a human being to find meaningful patterns in data we at iQuest Analytics believe that the greatest pattern matcher is still the human brain....at least for the next 15 - 20 years ;)

With iQuest BlueAnt we allow patterns in the data based on grammatical network relationships to assist the skilled knowledge worker to infer relevant meaning with context. By creating a rich semantic knowledge structure index we allow users to uncover more relevant information with ‘Contextual Search and Analysis with Intelligent Discovery’ for better sense-making that’s crucial for uncovering hidden patterns and relationships with actionable insights.

 

- Steve Ardire, iQuest EVP Strategy

People vs. Machine: the State of Enterprise Search PT 1

We at iQuest Analytics believe that the greatest pattern matcher is the human being. It is NOT the role of the machine to do that work for the searcher. It is NOT appropriate to impose bias into the enterprise search equation. It is simply foolish to believe that machine AI can approximate to the skill, accuracy and speed of a human being to find meaningful patterns in data or the natural environment. Infants, during the earliest period of bonding, rapidly identify patterns of behavior of the caregiver- a necessary precondition of early survival. 

We believe that iQuest can provide a system of search that will allow patterns in the data, based on grammatical network relationships, to assist the skilled knowledge worker in creating meaning and context.

Current Enterprise Search applications, models and architecture fall short of the promised goal of allowing users, from the least skilled to the most skilled, to find, in context, what is needed for the search to be successful. Some organizations I am familiar with have 100's of repositories, multiple context management, collaboration and MDM systems and generally lacking Enterprise Federated Search applications.

It is a fact that tens of  millions of dollars are spent on paying skilled knowledge workers to "manage" the meta layer and resultant onthologies and taxonomies. It would appear that such valuable resources are better served applying their expense skills in the service of moving the enterprise forward.

It is a fact that most if not all enterprise-wide search and discovery systems require continuos top-down management of the data, presuming that there is enterprise-wide awareness of where that data lives.

I am of the belief that given the right architecture and search philosophy, enterprise search based on a combination of NLP, grammatical network analysis and sound semantic modeling, all data, structured and not, can be accessible to the searcher without the initial requirement of the heavy cost of top down translation, normalization and ordering of the data.

Monday, October 13, 2008

Finding Meaning One Word at a Time

When analyzing textual data there is a tendency to go narrow - within a document- not wide - across sets of documents within a corpus. Historically, wide and deep has been expensive in processing power and human resources. No longer. BlueAnt incorporating the iQuest methodology adds speed and accuracy to the investigation process.

When analyzing a corpus of documents for meaning a user must go beyond what is contained in individual documents and investigate how documents are related by common terms, concepts, ideas and meaning. Sometimes, meaning appears only in the relationships among a set of documents, meaning which is not discernible in any individual document. Thus, an analytic tool must be able to guide the user into seeing document relationships. At a fundamental level, these relationships are the terms that are common among any given set of documents. Term level relationships can then be built into concepts, ideas and meaning relationships.

An example of this might be an intelligence analyst looking for preliminary indications of a terrorist event. It is extremely unlikely that the planned event will be fully described in a single document. But it is possible that pieces of information about this event are spread over many documents in the global data available to the analyst. If the analysis tool can guide the analyst to finding these related documents, it can be possible for the user to piece together a planned action to an event before it occurs.

 A very important tenet of the iQuest model is that meaning is found not only in individual documents but also, often more importantly, in the relationships among documents.

 


Sunday, October 12, 2008

Finding Meaning in Unstructured Data

Unstructured data is all the rage. There is so much of it, in company repositories and on the web. Making sense of it is the challenge at hand. Making sense of it without bias is a massive problem. I have been working on the problem for 4 years now and believe I have found a reasonable approach. My company, iQuest Analytics is releasing a grammatically driven search and discovery engine call Blue Ant. Blue Ant leverages indexed content of enterprise information repositories using a process that extracts relationships of grammar such as parts of speech, natural language processing and semantic analysis. Blue Ant lets the data speak for itself. Blue Ant elevates patterns within the content creating dynamic contextually linked data. I believe that no other unstructured data analysis tool can accurately make this claim.

Thursday, September 25, 2008

In Search of Meaning

It was Viktor Frankl's  central premise that we humans, at our core, continually search for meaning in our life. It is this search that helps define who we are and  what we become. This central premise takes on new meaning as many of us move our "second self", or avatar, on to the web. There is a grave danger of forgetting who we are as we strive to become a "second self" in the virtual world. Some might get lost in the translation and therefore become devoid of what truly makes them human.

To use a term coined by William Gibson in Pattern Recognition, our life on the internet is like living in a mirror world.  It is no surprise then that Web 3.0 is about semantics.  Semantics is the study of meaning. I suggest then that this study of meaning once personalized will be revolutionary in how the first self - that self living in the physical world begins to meld with the second self - our persona on the web.

Monday, September 22, 2008

Statistics and Loss of Humanity

Steven Baker's Numerati is a must read and a brilliant tale of mathematicians and statisticians that tract our every click. It tells the tale of the lords of statistics in graphic if not pornographic (depending on one's sensibilities) detail. We are exposed, manipulated, molested, accosted and tracked. The Numberati track us and our habits 24/7. These statistical lords identify our patterns of behavior in politics, love, shopping and war. To what end and for what purpose? To get us to act in a certain manner to convince us that we belong. To somehow redefine and define where we fit. To get a vote, to sell a product, to seduce.

 

Its 3 AM and I wake up in a cold sweat, I dreamt that we all became just a number to be counted and manipulated, maybe not for the purpose of extermination, but certainly for the purpose of dehumanization.

 

And I worry that a “digital self” is replacing the “human self”. Nothing more than a bar code and a mouse click.

 

The power of this technology, the ability to dig deep and discover could be turned on its head for the common good, not the good of the few.

Friday, September 19, 2008

the unintentional genius of the Palin choice

Whether unintentional or Machiavellian, McCain and his handlers threw a wrench into the rather predicable pattern of campaign season politics. Not only is a women put up as VP she is instantly positioned as martyr material as well. One only has to look at the general response to McCain's announcement. The genius in the pick is, although the response was WHAT? instantly setting up the media and many very thoughful people as the ENEMY of the first women VP candidate on a Republican ticket. Animal Farm from the start, pit bulls, pigs with make up.
From that moment forward everything went sideways. For most human beings, as primitive as we often are, having a target requires shooting at it. and shot we did. Not productive.
We were instantly pulled off message and into the age old game of noise over substance. If we stick to our core beliefs we can move beyond the carnival atmosphere that plays to the republican strong suit and circle past Palin as booby trap and win in November