Up Periscope!

It's been a while since I've posted, partly because I've been extremely busy with work and other projects. That said, someone forwarded me an article and I just had to provide some comments. Feel free to agree or disagree.

...


This article (URL below) presents a flawed suggestion that the word "crowdsourcing" is bad.

http://it.toolbox.com/blogs/database-soup/never-say-crowdsourcing-34331

Humorously, the article falls into its own trap. For example the word "exploitation" ... the author associated exploitation as a bad thing. Probably most of us do.

But actually exploitation has two meanings. From m-w.com 1: to make productive use of: UTILIZE - 2: to make use of meanly or unfairly for one's own advantage

Now the second definition is a bad thing. But the first one is neutral. You can exploit a sandwich to provide energy for your body (left by itself and uneaten the sandwich would not be of productive use).

James March, a rather famous author in organizational science, wrote of "Exploration and Exploitation" in 1991 -- his exploitation was the first definition, not the second. Productive use of existing processes or knowledge.That's not a bad thing.

Particularly because knowledge, unlike material goods, is something you can give away and yet still process.

... so ironically, this article about "Language Matters" trips up on its own premise when it says crowdsourcing is exploitation -- and then proceeds to assume exploitation is always bad. I'd use the tag !epic_fail but I sense the author was trying to make a point, but came across as rather narrow-minded in the author's own restricted view point.

Would the author be upset at Sermo.com -- where doctors are paid for their insights. That's "Crowdsourcing" within the medical community. That's exploitation. Is that bad? Or what about Threadless.com -- where designers of patterns ultimately produced and sold receive a royalty for their ideas. That's "Crowdsourcing" of t-shirt. That's exploitation. Is that bad?

On second thought, I will label this article as uninformed, ignorant of alternative definitions and instances, and making a lot of hay about nothing.

Semantic Bot Parsing for Localized Recommendations - Part 2

Following up on the earlier "Part 1" of this post, I also sense that Apple, Google, and others are gearing up for LBS and this space will get crowded with big players in time.

The idea though would be to do this pre-emptively and one-step ahead so it's something attractive for acquisition. Thus the idea of going beyond "websites" or portals and moving towards bots that search for you and pull data housed in the cloud so the data finds you (vs. you having to search for it).

Knowing what people are searching for at both a personal *and* localized level probably would be extremely powerful. Do this over time and you learn their tastes and pattern of activities.

Also agreed that writing a bot, coupled with natural language processing, wouldn't be too difficult if the inputs were coming from a Twitter / Wave or other short, truncated sentence. In some ways, we're simplifying our language so it's easier for the machines to understand us!

This may be something for a consortium or academic partnership to pursue... while this has commercial purposes, it also has collaboration and crisis response purposes. Bots that find important information relative to your role for you, bots that find likely people in government across agencies and across federal-state-local levels who do roles like yours and who you probably should be collaborating with, bots that show where the different response teams are and their status, etc.

... Wonder what it will mean when the number of "live" bots on the Internet surpasses the number of humans? (Then again, there are many more insects that humans, so maybe the better question is what does it mean when the number of Internet bots surpasses the number of all multi-celled life forms on Earth?)

Thoughts?

Semantic Bot Parsing for Localized Recommendations - Part 1

1. Scanning the space of Internet and web development, it seems like most folks are still focused on web sites. Sure, some have moved to "Web 2.0" apps that allow remixing of content, or *social* content, but in terms of radical change, it hasn't happened yet. By and large you still need to *go* somewhere online to search for information (vs. the information finding you).

2. Localization and personalization of information on the web represents, in my opinion, an untapped "value" added. Determining who you are by your Twitter/Facebook/Google profile coupled with *where* you are by geopositioning your IP address or, better yet, your GPS-based cell or car device, represents a chance to deliver timely information relevant to who and where you are.

3. Add to this the trend to expecting near-real time, asynchronistic, social interaction of the web (Google Wave, Twitter, Yammer, other microblogs) coupled with semantic/natural language processing raises an interesting opportunity -- web agents that chat with you and provide recommendations (without you necessarily having to go search for the info).

... You're already seeing this a bit, but not to the scale it could become. For example:

You and your friends are on Google Wave (or Twitter). You want to go out that evening but you're not sure what's going on around town. You can "include" a bot in your group of friends capable of semantic processing into your Wave (or Tweet). You say:

@travelbot0219 what events are happening in Clarendon tonight?

... The bot hones in to key words in your sentence:

[What] plus [?] = this is a question

[Event] = you're looking for an activity

[Clarendon] = correlates to a set of zip codes. There may be multiple Clarendons in the U.S. so the bot either has to see if it can identify your location by your Google Wave (or Twitter) profile or the geoposition of your IP address

[Tonight] = corresponds to a set. You're looking for events between 6pm until midnight associated with the time zone of Clarendon (that midnight time frame might be pushed further back to 2am or 3am if your Google or Twitter profile shows you are a teenage).

... And from this follows a decision tree as to whether or not it knows enough to answer your question. In this case, events is ambiguous and it should confirm Clarendon, VA as your location of interest, so the bot Waves (Tweets) back:

@coolkid59 what type of events in Clarendon, VA between 6pm-midnight are you interested in?

@coolkid59 example of event types include: movies, music, sports, arts?

You then reply back "movies and sports starting after 8pm" ... Which the bot parses as having types of events and notices the words "after" and "8pm" ... The bot also notices you didn't say "no" or some other form of Clarendon (e.g., Clarendon, TX) and concludes that it now has enough information to search for events in a database that the bot has rights to access.

This database contains the results of user-entered plus EDI-transferred plus web-scrapings of events around the U.S., indexed by zip code, date, and type. The bot can then answer with 3 events that match your search (as a Wave, Tweet, or Yam) and then provide a URL for additional details on a webpage customized to your query.

Of course, you can also do the same thing via a standard webpage search *or* via email/SMS messaging (useful if you're using your cell phone to spot events in your area).

How do you make money off of this? (1) Product placement. Folks can pay to have their events be one of the top 3 recommendations for the evening. (2) Ads on the website -- or, if they're using Wave, Twitter, or Yammer, ads on the URL link provided with more details... the URL link wouldn't be just a textual display of events, but a Google Maps display of the events, showing where they are color coded by type and allowing you use a "timeline" slider to see what events appear or disappear associated with your event.

As mentioned, you're populate the list of events by allowing any user to submit by a webpage, EDI-transfer, web-scrap (of Ticketmaster, ESPN, and other sites with event info), email/SMS submissions, or Google Wave/Tweet/Yam:

@travelbot0219 art show on 10/27 at 19th and Penn, starting at 6pm

@coolkid59 How long will the art show event at 19th and Pennsylvania
Ave. in Washington, DC on 27 October 2009 starting at 6pm last?

@coolkid59 And is the event free or are there costs?

@travelbot0219 it'll last until 10 and it's 15 bucks to enter

@coolkid59 Confirming 6-10pm for the art show at $15/person

(... The identity of @coolkid59 having already been confirmed to be a real person and not a spammer... Additionally the service tracks the reputations of submitters so if @coolkid59 has a high enough reputation, this pushes this event to the "top" of event recommendations.)

Lastly, the site tracks the taste of folks (akin to Netflix... Folks who enjoyed this event, also enjoyed these events... Or folks with similar Facebok profiles as you enjoyed these events) and might even play an optional "match-making" feature.

@coolkid59 Do you want to possibly go to the event with someone else like you?

@coolkid59 If so, are you looking just to hang-out, make a friend, or perhaps something romantic?

(... This might be a service where 'trusted' identities are brokered like Match.com, the site knows who the person really is ...)

And the service could follow-up with you the next day:

@coolkid59 Did you have a good time at the art show last night? How would you rate it on a scale of 1-10 (10 being the highest)?

... Cumulatively, these represent some of my ideas for a business that would be game-changing for the internet. If the bots prove truly useful, you could "sell" a subscription to a user for features beyond recommending events or local activities or match-up... This could include your own restaurant or hotel recommending and booking agent (also for flights and rental cars). Also a recommending agent for local sales and specials at shopping centers and grocery stores.

What are your thoughts? Might this be a possible start-up? Worth flushing out a larger business plan?


The bots would be housed on a server farm and need to interface, through API's, with Twitter, Wave, Facebook, etc. The servers would house the database of activities and the reputations of users submitting events (allowing events to be submitted via web, EDI, or other means) as well as the likes and "tastes" of people using the service. The server farm would also crawl for events and allow "product placement" (event placement) and ads to monetize on the business.


Semantic Bot Parsing for Localized Recommendations - Part 1

On-going Discussion Regarding "Knowledge Management Policy" in Government - Part 3

The following represents an on-going discussion regarding "Knowledge Management Policy" and whether encouraging more KM policy might result in a potential bureaucratic nightmare. My professional and academic experiences all suggest yes, but I'm the minority view at the moment among the discussants having the conversation...

Feedback/comments welcomed!

Hi xxxx,

I think the trouble with the KMgov effort is they're jumping straight to the "how" and missing asking the *why* that needs to be considered antecedent to how... because the *why* will reveal deeper problems than just KM (that a KM initiative could either resolve or exacerbate if done incorrectly).

So... what I'll (briefly) try to convey you are two logical arguments... that also detail how such a "cultivator/gardener" role would look operationally:

(1) that the current system of government as it is structured would preclude success of a CKO (even if they were well-meaning)

(2) that a successful KM initiative that remedies the structural problems in government doesn't require a CKO (because that role becomes ubiquitous)

First for (1)

Let's assume we were able to find the right CKO who understood her/his role as a Gardener. We put her/him in a high enough position (within the EOP) so they had a good visibility. Now, as a Gardener, the CKO is going to need to influence the efforts of departments and (hundreds) of Federal agencies. How does she/he do that? What levers can she/he pull?

... Influence over the budgets of Departments? Probably impossible, since members of the President's Cabinet would fight tooth-and-nail to retain all the funding they could and if this became a "budget review" function, even with the buy-in of the Cabinet, you'd encourage protectionism by the civil servants of each agencies so much that they "fudge" their numbers, go on a defensive posture, and just wait until the CKO's term of service is up (again, passive-aggressiveness is what the system currently rewards).

... So, we can't influence budget and achieve a successful transformation, what can the CKO influence? Definitely not the hiring or firing of personnel for Departments, that's a sphere again currently protected by Cabinets, and even if you could "embed" personnel of the CKO into Departments, they face the same problem of how do they influence events in that Department? What sway do they have? Answer: very little.

... Okay so no luck in either path, so let's step back and realize that even if the CKO could try and influence collaboration, the whole hiring/promotion of government employees rewards individualistic behavior -- not collaborative behaviors. A government employee's six-month and annual performance review is based on what that employee did, not necessarily what you helped someone else do (or even if this is factors, the someone who lead/did the effort sees larger gains than the helpers). So the CKO would need to encourage structural changes in how our government handles hiring/promotions, but that's already the domain of OPM. OPM should be working this angle.

... Moreover, the whole three-year project-agency-budgets-in-advance cycle that Congress and OMB follow also lacks any rewards for collaboration, transparency, or even *honesty* in the projected numbers. Right now OMB doesn't give "extra funds" to projects demonstrating that they help not only your department, but other departments or agencies as well. You have money left over at the end of the year? No rewards for giving it back or to another agency, but rather *penalities*. Instead, there need to be penalties for projects that only help your agency and, conversely, *rewards* for pitching projects that help multiple agencies (with multiple collaborators). Okay, so the CKO would need to encourage this change, but that's both Congressional and OMB turf. The EOP working with Congress should be working this angle.

So what I'm trying to show you is that massive transformation, above and beyond KM, is necessary for a CKO position to have hope of succeeding. This isn't just a problem for a CKO, it's also a problem for a cyber-security czar or ODNI czar or any government leader/cultivator who's trying to encourage inter-agency collaborations. The very system wasn't built for folks in their position to operate effectively, and structural changes above-and-beyond "KM policy" are needed first before their positions ever have the hope of working.

Now for (2)

... How might we remedy the structural problems in government, well, I've suggested some above so with proof (1), so I won't detail them again. Let's say in our KM Initiative, we spent the next 12-18 months working to detail these problems quantitatively and qualitatively, demonstrating how things aren't working as they ideally should and produce case studies so much that it becomes *glaringly* obvious to OPM, OMB, and Congress that change is needed. This is a "bottom-up" approach to produce enough of a crisis that political appointees have no choice but to address the problems. We produce the mini-crisis that necessitates political appointees approaching us and saying, okay, so what's the solution.

... Thus, in parallel, we start piloting mini-solutions, but instead of a massive, centralized effort, we do mini-pilot efforts (where possible) in our own agencies and across agencies -- requiring brave souls willing to risk a "safe and easy" government career and to work together, possibly at the expense of easy upward mobility. We try mini "inter-forums" or knowledge ecosystem efforts, and *share* (freely) this technology with other departments that are interested. We start practicing what we're advocating for a new way of government working together. With our group of true believers, we try to remove ego from the equation, instead advocating peer-to-peer (vs. hierarchical) relationships. Also, we submit inter-agency budget proposals to OMB suggesting additional funding because this benefits multiple agencies.

... Brave CIO's of agencies also start pushing their leadership and OMB to allow discretionary funding each year (say, 10% of their budget) to be spent on proactive, pilot projects for next-generation government... again with an eye to building from the bottom-up the future we want. Having already documented the problems earlier, now our KM Initiative group members collectively germinate several seed/pilot efforts in different domains, each year pruning and cultivating the more successful efforts and opening up successful efforts for reuse or expansion of use by other departments. But this will take brave governments willing to risk not being promoted because it "wasn't built by them" or "they didn't develop the solution" as well as cultivators of the pilot effort willing to "let go of sole ownership" of their creation and "share in the credit" with others.

... With all of this, such a bottom-up approach doesn't require a CKO, but instead a group of dedicated folks working from the grassroots-level to impact change. We continue to document the problems we're trying to solve in government, how they're structural and include the hiring/promotion/behavioral elements that OPM and OMB both need to address. And we also demonstrate proofs-of-concept that work for one agency, multiple agencies, and eventually reveal a solution that (if funded) could work for all of government. The funding comes when our documentation of the structural problems becomes too clear to ignore -- a "bubbling up" as it were.

... So, say 15 years from now, if government has undergone this transformation and now government workers are rewarded for collaborating with others in the basis of the problems to be solved and not on the basis of hierarchy or agency labels, and if budgets are done to reward addressing these problems, with some annual discretionary funds for emergent "new ideas" pilots as well as emergent problems or opportunity to be addressed, you now have to ask what is the role of a Agency "Director", what do they do with this next-generation form of government?

Well: they exist to cultivate networks of collaboration both within and across their agency to other agencies. They work to cultivate the best and brightest ideas and align budget funds to address those good ideas, emergent problems/concerns, as well as identified opportunities. They (the Agency "Directors") become the Agency "Cultivators" working together with other Agency "Cultivators" in a true peer-to-peer fashion. Hmm... that sounds a lot like what the CKOs we've been discussing would be doing. What I'm trying to suggest is that the agencies CKOs *aren't* needed if we become the government necessary for CKOs to succeed because the Agency "Directors" as "Cultivators" take up this lead role... at the EOP, the President is the uber-CKO. Moreover, individual government employees now are incentivized by the structure and behavioral rewards of the system to do the right thing generally, without the need for a distinct CKO separate from the leadership/cultivator-ship of this brave new world. We all (in government) become mini-CKOs working with other mini-CKOs for the betterment of government, like ants looking to identify sugar peaks (there is no CKO for ants, yet they're extremely adaptive to novel environments).

________________________________

... so, have I convinced you? :-)

(1) that the current system of government as it is structured would preclude success of a CKO as you detail below (even if they were well-meaning)

(2) that a successful KM initiative that remedies the structural problems in government doesn't require a CKO (because that role both folds into the role of Agency "Directors" and becomes ubiquitous)

Best regards,

-d.

On-going Discussion Regarding "Knowledge Management Policy" in Government - Part 2

The following represents an on-going discussion regarding "Knowledge Management Policy" and whether encouraging more KM policy might result in a potential bureaucratic nightmare. My professional and academic experiences all suggest yes, but I'm the minority view at the moment among the discussants having the conversation...

Feedback/comments welcomed!

Hi xxxx,

As background, Herbert Simon won the Turning Award in 1975, Nobel Prize for Economics in 1978, National Medal of Science in 1986, von Neumann Theory Prize in 1988, and APA's Award for Outstanding Lifetime Contributions to Psychology in 1993. Apropos to our discussions he began his career looking at decision-making in government and concluded:

"The human being striving for rationality and restricted within the limits of his knowledge has developed some working procedures that partially overcome these difficulties. These procedures consist in assuming that he can isolate from the rest of the world a closed system containing a limited number of variables and a limited range of consequences."

... i.e., bureaucrats in a bureaucracy only see the solutions that the bureaucracy presents to them as solutions. Think of it as a landscape where you're on a small hill. You look all around you and you see a valleys and a mountain range that surrounds you... because you can't see over those mountains, you don't see that there might be a more green, richer, lusher hilltop over the mountains. You're bounded by the environment that surrounds you, in this case the government bureaucracy, and so you think the only solutions in your "toolkit" are bureaucratic ones.

This is why I continue to suggest we can either operate with the confining mid-20th century government environment (that has produced the very problems we face that require KM in government) or work to change that environment.

Why do we think using "old tools" that caused the problems we're trying to solve will help us now? Creating yet another centralized agency accompanied by top-down policy are 20th century bureaucratic solutions, and represent metaphorically that limited hilltop perspective that Simon describes. Instead, I'm recommending we try and rise above that, citing some pretty famous individuals who have employed empirical studies to suggest the same thing.

... Also, I respectfully disagree that INTRA-agency should be the focus. Sure, it's a problem and sure, the government should also work to fix that, but each agency Director has a direct sphere of influence in which they can shape the environment within their agency to improve collaboration. If they can't shape items in their own agency, then they should *not* be Director. The bigger, 21st century problems we collectively face require INTER-agency solutions; to wit:

* Another Hurricane Katrina <-- that's INTER-agency

* An effective U.S. Intelligence Community <-- 16 different Federal agencies and a *massive* budget

* A terrorism event on U.S. soil <-- that's INTER-agency (requiring over 90+ different Federal agencies to work together)

* H1N1 continued spread <-- INTER-agency (over 200+ Federal, state, and local agencies to work together)

* Climate change <-- INTER-agency Continued economic decline and attempts to trigger a rebound <-- INTER-agency

INTER-agency, no one but the EOP technically oversees the inter-agency space. This is where we need to assist because there's a vacuum at the moment. Sure, along the way we can provide suggestions for improved INTRA-agency collaborations, but that should evolve and be unique to each agency and the Director (if they're good at directing) has the ability to cultivate that environment. It's the INTER-agency one that the 21st century problems we face require a fundamental re-think of how our government works... together.

<\two_cents>

p.s. One last quote from Herb Simon: "The world you perceive is drastically simplified model of the real world." :-)

-d.

On-going Discussion Regarding "Knowledge Management Policy" in Government - Part 1

The following represents an on-going discussion regarding "Knowledge Management Policy" and whether encouraging more KM policy might result in a potential bureaucratic nightmare. My professional and academic experiences all suggest yes, but I'm the minority view at the moment among the discussants having the conversation...

Feedback/comments welcomed!

Dear xxxx and xxxx,

Though I respect a lot of what you've said, afraid I have to respectfully disagree with the specific comments pro a centralized, top-down policy approach. If I were to recommend one book for this discussion it would be "The Science of the Artificial" by Herb Simon:

www.amazon.com/Sciences-Artificial-Herbert-Simon/dp/0262691914

... now, as for why I disagree:

(1) Government is an organization of human beings, and therefore can be whatever we want it to be. It can *be* a bureaucracy, but there are no fundamental laws of the universe that says it has to be as such.

... As a bureaucracy it has merits and weaknesses, but I disagree with anyone who says government is inherently a bureaucracy. We could be a monarchy or a dictatorship if we wanted to be (but fortunately, we don't). We can either try and work within the confines of a pre-defined environment (in this case the current bureaucratic government) or work to change that environment.

(2) I suggest that the problems that our Initiative attempts to address and the problems that require KM in government stem from the *very bureaucracy* that was created as part of its evolutionary design.

... The bureaucracy was not designed to encourage collaboration, the designers of the bureaucracy did not envision the government needing to respond rapidly or solve problems that span multiple departments and agencies. The designers of the bureaucracy wanted to reward protectionism and empire-building because this was a way to prevent any one part of government from becoming too powerful (ambition to counter-act ambition)

(3) Thus, if the reason why we need KM stem from the bureaucracy that was created... why do we somehow think that a bureaucratic solution would remedy the existing flawed environment that necessitates KM?

... The problems we're confronting are structural and behavioral. I'm not saying we won't need policy to complement the larger transformations needed to make this happen, but policy is an often-used-and-abused lever which I have not seen used (yet) to address inter-agency, complex problems in government successfully. I'm open to someone demonstrating a recent example that can prove this statement wrong.

(4) If our current government bureaucracy was designed to have "isolated islands" and for divisions within agencies not to have incentives to collaborate, or agencies within departments to have incentives to collaborate, why do we think a bureaucratic solution will somehow *fix* this? We're going against the very design of the system.

... Again, we can either try and work within the confines of a pre-defined environment (in this case the current bureaucratic government) or work to change that environment... I suggest we work to change that environment through the very way that we are presently: grassroots transformation.

(5) Ergo, I suggest that we need solutions that recognize that top-down government (through policy) works only when you're not trying to span across different groups and address complex problems... I can testify both as an academic who has researched this in several U.S. government agencies and as a practitioner currently working with the U.S. and military forces in Afghanistan: top-down approaches will fail when addressing complex problems.

________

p.s. If I were to suggest a lofty goal for our KM effort, it would be to recognize that we're really advocating something much more transformative: a shift from bureaucratic to a post-bureaucratic form of government necessitated by the global challenges and opportunities we now confront. With this, I'll end with one last book recommendation: "The Post-Bureaucratic Organization: New Perspectives on Organizational Change"

www.amazon.com/Post-Bureaucratic-Organization-Perspectives-Organizational-Change/dp/0803957181

... I'm not saying we may not need policy to complement the large transformations we need to do to make this happen, Again, we can either try and work within the confines of a pre-defined environment (in this case the current bureaucratic government) or work to change that environment... I suggest we work to change that environment through the very way that we are presently: grassroots transformation.

-d.

Science of Security

Currently at a series of day-long meetings with NSF, NSA, and IARPA folks regarding whether or not we can create a "Science of Security" (focused on cyber-issues) and what such a science, if possible, would look like. Interesting discussions and many different perspectives, ranging from hard-core experts in computer science, physics, biology, and sociology.

My own thoughts, for now, are probably baised by the fact that my background includes both cyber and a biology background. I think any science, if possible, would have to draw from complex adaptive systems (CAS) theory. I ask these thoughts similar to the discovery of the “Omega” and “Super-Omega” numbers (see article below). Could it be, with cyber-security efforts, we are facing a complex environment that includes complicated behaviors bordering at times on randomness that occasionally presents itself as having links and reason; that is, that the default state is “anarchy, not order”? (Again, see article below for more on how mathematics has recently revealed itself to be riddled with holes and anarchy).

This is not to say we shouldn’t engage in the activity of trying to assess a Science of Security, but approach it with the realization it will never be complete (even at the high level) and it can only be a reflection of what we can best imagine, envision, and potentially know with the purpose being, as best as possible, pro-actively positioning resources to try to reinforce the security of such systems as well as be ready to respond when/if a system is attacked, compromised, or crashes?

Thoughts?

Excerpt text re: Omega and Super-Omega Numbers
http://www.newscientist.com/article/mg17022856.400-omega-man.html

Gregory Chaitin, a mathematics researcher at IBM’s T. J. Watson Research Center in Yorktown Heights, New York, has shown that mathematicians can’t actually prove very much at all. Doing math, he says, is just a process of discovery like every other branch of science: it’s an experimental field where mathematicians stumble upon facts in the same way that zoologists might come across a new species of primate. Mathematics has always been considered free of uncertainty and able to provide a pure foundation for other, messier fields of science. But math is just as messy, Chaitin says: mathematicians are simply acting on intuition and experimenting with ideas, just like everyone else. Zoologists think there might be something new swinging from branch to branch in the unexplored forests of Madagascar, and mathematicians have hunches about which part of the mathematical landscape to explore. The subject is no more profound than that.

The reason for Chaitin’s provocative statements is that he has found that the core of mathematics is riddled with holes. Chaitin has shown that there are an infinite number of mathematical facts but, for the most part, they are unrelated to each other and impossible to tie together with unifying theorems. If mathematicians find any connections between these facts, they do so by luck. “Most of mathematics is true for no particular reason,” Chaitin says. “Math is true by accident.”

This is particularly bad news for physicists on a quest for a complete and concise description of the Universe. It’s a bitter pill to swallow, but even Steven Weinberg, a Nobel prizewinning physicist and author of Dreams of a Final Theory, has swallowed it. “We will never be sure that our final theory is mathematically consistent,” he admits. ... Omega is infinitely long and utterly incalculable. Chaitin has found that Omega infects the whole of mathematics, placing fundamental limits on what we can know. And Omega is just the beginning. There are even more disturbing numbers—Chaitin calls them Super-Omegas—that would defy calculation even if we ever managed to work Omega out. The Omega strain of incalculable numbers reveals that mathematics is not simply moth-eaten, it is mostly made of gaping holes. Anarchy, not order, is at the heart of the Universe.

[... excerpt ...]

But Chaitin already knew that each digit of Omega is random and independent. This could only mean one thing. Because finding out whether a Diophantine equation has a finite or infinite number of solutions generates these digits, each answer to the equation must therefore be unknowable and independent of every other answer. In other words, the randomness of the digits of Omega imposes limits on what can be known from number theory—the most elementary of mathematical fields. “If randomness is even in something as basic as number theory, where else is it?” asks Chaitin. He thinks he knows the answer. “My hunch is it’s everywhere,” he says. “Randomness is the true foundation of mathematics.”

[... excerpt ...]

The fact that randomness is everywhere has deep consequences, says John Casti, a mathematician at the Santa Fe Institute in New Mexico and the Vienna University of Technology. It means that a few bits of maths may follow from each other, but for most mathematical situations those connections won’t exist. And if you can’t make connections, you can’t solve or prove things. All a mathematician can do is aim to find the little bits of maths that do tie together. “Chaitin’s work shows that solvable problems are like a small island in a vast sea of undecidable propositions,” Casti says.