Image source: Wikipedia

This is going to be slightly off topic, so you’ve been warned. Carly Simon seems to be bubbling up into pop culture consciousness, even without being included on a Quentin Tarantino/ Wes Anderson film soundtrack. Her controversial “He Hit Me” she wrote that is covered by Grizzly Bear on their Friend EP and in live shows.(Please read comment thread.) On an anecdotal level, a friend, Ray, randomly mentioned that he has been into the Simon track “Why.”

A little additional internet research is showing that she is also releasing her next record on the Starbucks label, and one of her songs was featured on American Idol. I’m waiting to see if she shows up anywhere else. How are these related, if at all? Can the fluctuations of the comeback popular culture be tracked, traced, understood and gasp predicted?

All this leads up to the big question I want to ask, which has to do with her ode (dare I almost say, pre-rap battle?) to Warren Betty (or Kris Kristofferson or Mick Jagger, depending on who you ask) “Your So Vain.”

The lyrics of the accusatory chorus “you’re so vain, you think this song is about you, don’t you?” have always confused me.

um… the song IS about him, isn’t it?

Frank pointed out some clever research at Microsoft Research. Eric Horvitz and Jure Leskovec parsed through one month of MSN messenger communication, or about 1 billion conversations a day. Among the 240 million users, they discovered an average of 6.6 degrees of separation between any two random users. 6.6 is obviously close to the famed six degrees of separation found in Milgram’s 1967 study. Although, some debate still continues on the validity of that finding. Horvitz made the full paper available, and has really in depth analysis of the spread of MSN Messenger and the communication it facilitates. The image above shows the density of users. The numbers of user shift from high to low according to the light spectrum, with red as high and blue as low (think ROYGBIV.) Since this finding, Horvitz wonders if there is some larger phenomenon at work, with six being some natural average of social interconnectedness. More thoughts to keep me up at night.

Via (Roland Piquepaille’s Technology Trends)

Figure 1. Frequency from 2/2004 to 2/2008

Figure 2. Frequency from 4/2005 to 2/2008

Every once in a while, I get the inclination to try to do a little math. It’s a dangerous endeavor, but sometimes I can’t help myself. But before I get ahead of myself, I should give a little back ground.

Sometimes you learn a word and all of the sudden you see it everywhere. The nagging question is, was is always being used and you just glossed over it, or is there a change in the frequency of its use? You discovered the word precisely because it was starting to be used for more often.

Not too long ago, I found the word “awesomeness” entering my general vocabulary to signify great approval. Awesomeness feels fresh, a slight tweak on the new vintage “awesome” from the 80s, a personally influential era. However, in the past week, I’ve seen the word Awesomeness appear in a lot of places, from friends and strangers alike. I started thinking if the word has gaining traction in the public at large, so last weekend I started the quest to figure out ways to capture the growing use of the word.

Word frequency counts is nothing new. Media studies have been doing this type of research in newspaper and magazines for decades. However, it is becoming much more democratic, mostly because of the decreasing cost of computing and the related increase span of the web, which has been collecting data much easier. I decided to use the blogosphere because it seems like a pretty good proxy for general language usage. As well, the Google blog search feature allows you do set the dates for your search. Of course there are drawbacks to uses the word count of Goggle, including that the blogosphere is a obviously a subset of general usage and also because I have no idea how Goggle tracking and tallying the blogs it is indexing. Nevertheless, I can live with the approximation, given that the only cost for retrieving the data is bandwidth and time.

I searched for “awesome,” “awesomeness,” and “are” for each month from February 2004 to February 2008.

The word “are” was used as a control of sorts. Because the blogosphere itself it constantly growing, the number of times a word appears in a given month is expected to increase. Using an often used word such as “are” can be a proxy of the overall growth of the blogosphere. One could not expect the rate of the word “are” to fluctuate greater from month to month. Any true increase of the word usage would have to outpace the growth rate of “are.”

I also tracked the usage of the root word “awesome” for a couple of reasons. Sometimes search engines clump different variations of the same root word together in its search results. I wanted to check to see that “awesomeness” wasn’t being put together with “awesome.” The two are also an interesting comparison. If both increased at similar rates, then maybe what I am seeing is just an overall revival of 80s idioms. However, if “awesomeness” is also increasing at a great rate than “awesome” my original suspicions would be validated.

In the short time I started on this little math adventure, the word kept on appearing, and in the write up of my findings, I came across the the ultimate reference, apparently a website which declared March 10 (last week) International Day of Awesomeness.

Just looking at the words appearance in the past two full months, this is what I found:

February-2008: Awesomeness: 17,182 ; Awesome: 736,783 ; Are: 61,531,049
January-2008: Awesomeness: 9,627 ; Awesome: 429,769; Are: 57,214,958

There is clear jump in the past two months, but what does that jump mean, if the total number of blog pages continues to grow? Both Awesomeness and Awesome almost doubled as compared to Are, but how do you measure the significance of that? Graphing all the frequency of these three words against each other is hard because are orders of magnitude higher then the others. My math coach Pam (yes, I actually call her that) suggested I take the log of my data to make it more comparable. If your recall high school math, log (1,000,000) = 6, log(100,000) = 5, and log(10,000) = 4. Now, if you take the log of all your data points, the curves can fit on a single graph of manageable size. It even gets better, because it translates an exponential curve into linear curve, which makes finding the growth rate (i.e. slope of the curves, which is the rise over run of the function) much easier.

If you look at the two figures, you’ll notice an upwardly trend. There is a peculiar elbow in the spring of 2005, which could be a big spurt of growth or some aberration of Goggle’s indexing. After looking at the first graph, I decided to draw another graph to focus in other growth of all three term’s use in the past year and focus the analysis on that because I wanted to fit a linear line to the curves and removing the bend would give me a closer fit. (Is that cheating?)

I fed the curves into excel to fit linear functions, and can see that Awesomeness has a slope of 0.0003 versus Awesome which has a slope of 0.0001. This is good, because it means Awesomeness is being used at rate that is 3 time more than Awesome. However, Are (our base line) has a slope of 0.001. This is sort of bad, because 80s slang doesn’t seem to be outpacing the general growth of blogs, which I was hoping to see.

I’m not sure what to make of it, in the end. However, it does have me thinking about blogs from a higher altitude and that math is pretty awesome. Many thanks to Pam and Wojciech who gave me some good nudges. Of course, I’ll take the blame for the conclusions. I’m curious to hear what my math friends say, especially if they find mistakes in my logic. Also, it has taken me much too long to post this, which is why I’m just throwing what I have up. I’ll post any corrections later including typos.

I have IM accounts on AIM, iChat, Yahoo, MSN, Gmail and Skype. (Years ago, I once used ICQ and IRC.) I always find it interesting, how certain services are popular in specific countries. AIM is most popular in the US, where as, MSN more widely used in Asia, South American, Canada. IM, like all networks, benefit from network effects. If a service gains traction early in a country, when it can maintain growth, even if it is surpassed by other services who may be globally larger.

The shrunken image above from Valleywag shows which social networks are most dominant by market share, which was tipped off to me by hellowojo. The color coding is a bit confusing, because they decided to make match the countries by the color of the social network’s logo. Because most of social networks logos are blue, a network’s reach is difficult to differentiate. Fortunately, Valleywag also includes another graphic with a list of all the countries by social network.

Going forward these data will change. I wonder if there will be a consolidation of these networks, start into interconnect, or with they stay fractured, in the same way that email addresses were once closed. Here are a couple of possible of future scenarios.

Just as standards for email have been created, Open Social or some other social networking set of standards can allow for competing sites to share information with each other. Wordpress or Moveable Type could become the open platform, where people host their own blog which would act like their profile, or use a service provider to maintain their profile/blog. Gigaom has postulated Wordpress’ move into social networking. Third party widgets could be created to offer function, such as status updates, photo albums, walls, and gifting applications to mimic many of the features of Facebook, LinkedIn, MySpace and others SNS. This sounds like a lot more work for the end user, which would be fine for the people who want to have control and heavy lifting which that entails. For the Blogger and LiveJournals users of the world, adoption is only going to take place if setup is as easy as creating a Blogspot or Facebook account.

Where the first example envisions some combination of blogs and SNS, another example is email fully integrating with social networking profiles. Google has been able to effectively enter the IM space by introducing a closely integrated chat client into its Gmail service. By sidestep the application download step, millions of their email users instantly became IM users. Similarly, the Xobni Insight plugin for Outlook, connects information profiles on email contacts, giving Outlook an SNS feel. I haven’t used Outlook in a couple of years, but if I did, I would certainly be trying to get a copy of Insight, which is still in an invite-only beta.

The final possibility to consider is the consolidation of social networking sites into one main site, which may have an open API like Facebook, but is also closed and propritary in the sense that people cannot easily export their profile data out of Facebook, MySpace, or Orkut. While people can leave at any time, if all your friends and contacts use a network then there is incentive to stay in that network. In this way, a company like Google, Yahoo, and MSN could use its adjacent email or blogging services to leverage its entrance into social networking and possible become the de facto platform. Although, as of yet, none of these sites have been about to make a major impact across continents.

So, I guess we’ll just wait and see what happens.

Image source: flickr

David Weinberger gave a description of a link in a panel last year at the Hyperlinked Society Conference. A link is a conscious act of generosity. These acts is moral, and they form the architecture of the web. He goes on to explain that the syntax of a link (i.e. the href HTML tag) has no meaning within itself, it is merely an instruction which points to another location. The meaning of the link, which can be agreement or disagreement, is found in the text surrounding the link.

While these links have no meaning, they do have value, which is the reason by creating a link performing generosity. Google ranks pages by the number of links other sites point to a page. Appearing early in a search result clearly has value over a later listing. You can only have a reputation if other people can find you. A page and her owner’s reputation then relies on the generosity of others linking to her page. If an author disagrees with the contents of page and wishes to dispute it, linking to the page adds to its value and reputation. The author is then left to not link. However, this practice which the status quo forces people to use still leaves the reader at a disadvantage.

There have been suggestions to create a newer kind of syntax and link taxonomy which would add to the current binary options of link or no link. The simplest system would be to have three choices, positive link, negative link and no link. This system would actually be very easy to for users. All you need to do is add a tag to the link.

Flipping forward one year, I was struck when Jonathan Zittrain pointed out in his talk last Saturday, the use robot.txt files for telling search engines not to spider a file or directory started in the early age of the web as an adhoc measure by individual which became an internet standard. Today, it is much harder to get a standard adopted, but the story of robot.txt reminds us that it is possible to create grassroots change in internet standards. Endorsement links allude to aspects of the Semantic Web, but frankly, I’m not sure if it will every come. Contextual syntax might evolve over time with gradual implementations.

The idea of rated links get even more interesting when you consider how search engines might use links that interpret reputation and authority. Of course, gaming the system would occur, but that happens now and should not deter the implementation of a link taxonomy. It might also encourage search engines to become open to annotating listings, as Frank Pasquale has suggested. Generally, search results are given by relevance or time of creation. New categories could be ranked in terms by agreement, disagreement or even controversy. The end result would be better ways for author to link, for readers to under the context of the link, and for searcher to usage links in the aggregate.

Image source: USGS

Yesterday, Facebook was frequently mentioned at the Symposium on Reputation Economies in Cyberspace conference, but I’m trying to figure out what is the value my Facebook network.

For a free service, Facebook is getting expensive. Not just for Microsoft, but for the users who maintain their social network. Dealing with Pokes, Invites and Scrabulous take time, effort and bandwidth. As the popularity and membership of Facebook increases, the cost for not participating grows as well. Just as there are costs associated with not having a telephone or email address, the social and economic pressure to join these sites can be readily felt.

These Facebook clicks of “friendship” are simple gestures that replace deeper interactions. This form of communication is low bandwidth, in terms of data but more importantly of cognition. We are now able to easily increase our social networks in terms of reach, but the fidelity of our interactions within the networks is decreasing. Facebook seems to value to the size of the network, but not the fidelity of the links. The value of a network is not only the number of nodes, but also the quality of the information that flows through the edges between the nodes. Finally, the work of Ronald Burt suggests that there is value in having a network that is unique from those with whom are competing. The low fidelity of Facebook communications show a shift toward networks which have low costs, effort, and unique characteristics, which overall have less value than we suppose.

Looking at usage rates, it is becoming the preferred tool for many people, young and old. One of the main reasons Facebook is a popular because of its convenience. We are able to maintain these relationships, which seemingly take a minimal amount of effort. A simple click allows us exchange gifts, play a game, or say hello. It also increases the efficiency of the user by automating one communication effort across many friends. By filling one movie quiz, we are able to apply this work to all our other friends who answered the questionnaire describing how much they liked “Shrek.” In a way, it’s like sending out a mass interactive Holiday Letter, which is admittedly better than nothing, but not quite fulfilling. Nevertheless, these Facebook apps are extremely efficient for members who have hundreds of friends. We are able to interact with all these friends though one gesture. For many current and recent students, Facebook is an intimate part of their social experience. However it is successful because it is compliments real time interactions, of a past history of deeper real time interactions. These compliments could have been face to face or some other digital communication form that has more fidelity than a Facebook post of ten word sentence on a user’s Wall.

What is the meaning of these gestures? What is the meaning when an app is flawed, as in the case of the movie taste matching application, if none of my favorite movies are listed? What is the cost of forgoing communication with higher fidelity?

The adoption of Facebook shows our willingness to extend a network (adding nodes) in exchange for quality of information and even meaning. Facebook is an important tool for maintaining relationships, especially when a person in a friendship moves away, such as attending a different university after high school or leaving a job. Before, these ties may have dissolved, but now they have a longer lifespan. But how long can a purely Facebook relationship last within itself?

The beauty of Facebook, and one of the reasons for it adoption, is that we can fake friendship, which is to say, simulate a relationship with minimal work. Rather than actually having a meaningful exchange with a person, in a couple of clicks you can send the latest application interaction to her. But has anything meaningful been communicated?

We do not acknowledge the trade offs between a large social graph with less fidelity in communication and a smaller social graph with higher fidelity in communication. With high fidelity, and more information, the relationship and connection will most likely wither away. This trade off is important and often left unconsidered.

Things just got really interesting in the mobile / wireless world. Verizon announced that they will over two categories of service by the end of 2008. One will continue to be its bundled handset service, and the other will be open to any device. This change brings Verizon Wireless in-line with the open networks that are available in Europe and Asia. The move will force T-Mobile, Sprint, and AT&T to consider offering similar services. (T-Mobile is already experimenting with allows users to make WiFi calls.) This announcement also may affect the upcoming 700MHz spectrum auction, as the FCC did not require open networks. I’m hoping that it will spur more innovation in the mobile space. Changes could happen quickly, we’ll have to wait and find out. I’m trying to stay optimistic.

Update: Techcrunch and GigaOM weigh in on the issue. A lot can happen in a year, and GigaOM is correct to be skeptical.

Images source: Wikimedia Commons, Matt Brim

The October issue of Discovery magazine has an article that piqued my interest, entitled, “This Man Wants to Control the Internet. And you should let him.” The man is Caltech professor, John Doyle, an expert in control theory. His field models dynamic physical systems, which includes things from a mechanical heart to space flight. The key idea is achieving a desired or steady state for one of these systems by taking current information about its state, and “feedback” that information to the system to make adjustments. These feedback system are mathematically modeled. When the system is non-linear and dynamic, for instance a airplane flying through wind currents, the mathematics required become quite sophisticated.

Doyle and his collaborator and fellow CalTech professor, Steven Low, have developed an improved protocol over TCP (or Transmission Control Protocol.) TCP describes how packets of data should be delivered and received over the Internet. FTP, email and WWW applications all rely on TCP. Using control theory, their protocol, FastTCP^TM, clocks the time a data packet takes to get to a final destination and make adjustments to optimize its stream of packets. Standard TCP does not take this extra information into account, and relies mostly on a strategy of monitoring lost packets. That is, packets that don’t make it to the finally destination. In the 2006 Supercomputing Network Bandwidth Challenge, they won it with a maximum throughput of 17 gigabits (a full-length movie) per second.

Improvements to the Standard TCP will be important in the coming years, as multimedia services (such as movies on demand) will increase the demand of the current network. Already, VOIP services do not use TCP, because packets sent using TCP cannot be received and sequenced fast enough for real time applications like phone calls.

Doyle and Low, along with Cheng Jin formed the startup, FastSoft, to sell products based on FastTCP^TM. However, they have trademarked their name and have submitted patents their technology. This is an important departure from the origins of the Internet, as no one owns that Standard TCP. Having to license or buy FastTCP^TM from FastSoft has implications to the future of the Internet, which could lead to its fragmentation.

Last month, at team from Indiana Univeristy, the Technische Universitaet Dresden, Rochester Institute of Technology, Oak Ridge National Laboratory and the Pittsburgh Supercomputing Center won the 2007 challenge. They achieved a peak transfer rate of 18.21 Gigabits/second and a sustained transfer rate of 16.2 Gigabits/second. It is not clear to me what kind of IP, the team from IU has on their technology. However, the received funding from the NSF, which may mean place of some or all of their research into the public domain.

Demands for bandwidth are only increasing. A complete overhaul of TCP is years ago, and involves incremental change, because the network at stake (that is, the Internet) is so important, which Doyle explain the Discover article. How we meet those demands is already controversial.

Susan Crawford notes that Comcast is already traffic shaping bits, by flagging packets by people using BitTorrent. (She also has a nice description of TCP in this post.) Meeting this growing need, the network can improve performance in various ways including: upgrading the infrastructure, such as laying fiber optic cable; improving data compression algorithms, and improving the protocols that control data traffic. In all these areas, the ownership and regulations of these technologies have huge implications on accessibility and adoption of the Internet. Although the Discover article’s title “this man wants to control the Internet” is a play on Doyle’s field of study, it raises an important point. Having public and private protocols may not only make parts of the inaccessible to each other, but further increase bandwidth as another form of economic inequality.

I’ve been slowing making my way through a very good book “Innovation and Incentives,” by Suzanne Scotchmer from UC Berkeley. I’ll close with a quote from her chapter on “Networks and Network Effects”:

“The protocols of the Internet and worldwide web were developed at public expense and put into the public domain. Given what turned out to be at stake, that is probably one of the most fortunate accidents in industrial history.”

Image source: IEEE

Noah Brier posted an interesting link recently to an article claiming that Metcalfe’s Law, which famously has been paraphrased to be the value of a network exponentially quadratically grows with each additional node, is wrong. This is a provocative thesis because the law is widely trumpeted in all that is good about the Internet.

In the theoretical long term view, networks with more nodes will encourage late adopters to join (and pay.) People often cite the instance of email, the rates of people setting up (and paying for) email accounts in the 1990s rose much faster after it was possible to communication with people outside a user’s ISP, being AOL, Compuserve, Prodigy, or an academic institution. However, network adoptions plays out differently in other cases.

The disconnect here may be that in the definition of value, more specifically value by whom. In the valuation of a network, the network of 100 is worth more than a network of 10. According to Metcalfe, it’s worth 100 times (10^2) the smaller network. For the consumer, it is clear that the value of the telecommunications network grows with each new person, especially if the costs for the user doesn’t increase. However, consider combining access of two competing networks from the perspective of the network operators. From this angle, the smaller network will clearly see more benefit than that larger network, as the article suggests. Further, both operators will see an increase in the costs associated with network traffic (from hardware to customer service,) with no new paying customers. If the operators are charging an unlimited usage prices, there is little upside to combining networks, which shows why network operators tend to resist interoperability. Why would a market leading operator take on additional cost with increased revenue and help a smaller competitor?

Coming back to the definition of value, the value of the network actually doesn’t change if the willing to pay (i.e. the ability to extract fees) from the customers of the network. I’m still trying to grapple with the difference between the value of the network from the customer versus the operator. That is, the problem is that as although the value of a larger network may grow, there willingness to pay by the customer does not. In fact, it may even shrink, which was noted in a MobileCampNYC talk by some smart folks at Air Arts last Saturday. Consumers have been conditioned to expect prices to fall, especially in the area of telecommunication services, even as nodes (and the value of the network) increases. Is the reason that customers don’t want to pay for that additional nodes because the new nodes are less valuable to the customer as the article suggests? Is it a matter of marketing conditioning from other services industries associated with the properties of the economics of scale that prices for services should go down over time?

Although the article was published last year, the valuations for networks will only receive more scrutiny after Facebook’s USD$15 billion valuation, which is about 25 times what News Corp paid for MySpace, which has more than double the users than Facebook. Expect more discussion to follow.

Here is a photo I took earlier this year at the then brand new Bangkok International Suvarnabhumi Airport (BKK). I was thinking about the image today, while doing some reading on social network theory. Airports are transportation hubs and feel weird when they are empty, because they are not acting out their function as place of transit. Shininess didn’t help matters at all either.

Some new points have instant connections while others don’t. I’m interested in how new points form and get connected into a network. Some points appear with a social network in place, say a baby born into a large family. Other points have connections which exist in other networks and their entry into a new network gives them instant connections as well. Steven Colbert made 1 million facebook friends in a week, which shows spill over from other networks. This is all obvious, but I’m curious if there are ways to show how different networks interrelated.

I will add more here, but I’m late for dinner.