Musings on the State of the Art

In the last post, I discussed some of the challenges around creation, acquisition and management of content. This post I want to look at the problems with distribution.

Content syndication has been around for a while for most types of publishers. There’s a lot of ways to consume information, away from its source of creation. These include RSS readers, aggregators, etc.

An example of feed flare for publishers

However, this hasn’t really been the case for e-commerce. Content syndication has thus far been limited to Comparison shopping engines, and in some select types of content (such as reviews) the content is aggregated together by a third party such as Buzzillions.

Why is this?

Shopping (especially in certain verticals such as fashion, home, and for almost all purchases above a certain ticket price)  is a social activity (whether you’re consulting your friends, your wife, your bff etc.) Yet this social component of the shopping experience has been slow to move over to the web. For the most part e-commerce is a one to one experience – and its tough to incorporate a social component into it without calling your consultant over to the screen to look at the product you’re looking at. This makes, “distributed” shopping very difficult.

Only recently have things like “share on facebook” started to pop up on sites such as Dr.Jays (fashion) etc.

Share on Facebook on sites such as Dr.Jays.com

This content syndication, and sharing is still in its infancy. Much more is possible, as should be obvious by taking a quick look at the publisher’s toolset.

As companies like BestBuy start to create open API’s to their data and structure and format of e-commerce data starts to follow a standard, we will see the democratization of content. Take a look at what Bestbuy offers below :

Best Buy offers tons of product related data

As you can see – it includes data for historical, and current products including near real time availability information for both online, and retail store locations. This is merely a first step.  Imagine the types of applications that could be built using data like this from all vendors? People “rolling” their own comparison engines that are limited to a few stores, and a few verticals that they are interested in. People getting RSS feeds of all products reviewed by at least 50 people that have a rating over 4 stars across all vendors, but limited to products that are in the category “Vespa Accessories”. All these slices, and dices should be possible today, but aren’t because of this limited access to data.

Sites like Fixya or RateItAll are trying to build this meta social layer, but they struggle with the basic information on each product – descriptions, specifications, features, images, manuals, etc. from the manufacturers and availability, pricing, shipping, and other value added content from the retailers.

Fixya.com attempts to build a social support layer for consumers

Once we have access to complete, well formatted, standardized data – what would be the effort involved in taking the next step and building the ability to socially share information about it. Imagine an application that makes shopping as social as PhotoPhlow makes photograph discussion. If you haven’t seen Photophlow yet – watch the video below.

This ability to discuss, share, talk in real time, or asynchronously will lead to a radical change in the way people interact with e-commerce sites. As data standards start to form, more and more content will be user generated. We can already see some of this in the form of user submitted pictures on Amazon, and certainly in reviews and ratings.

Images with notes as submitted by users on amazon.com

This data format standardization and availability will enable the widespread syndication, and sharing of data, and the merging of the manufacturer, and retailers supplied data with user generated data (the beginnings of which can be seen above). This in turn will enable a wide set of applications developed to help people shop in a wider variety of methods. At this intersection of widespread disemmanation of data, and social interaction between users, lies a gold mine of analytics data that will help surface, what people are buying, what the trends are, what the gating factors are – as well as who the mavens, and connectors are across the entire industry. I for one am excited to see it coming.

What else do you think we could build if we had easy access to clean e-commerce data?

Content has traditionally been a struggle for e-commerce vendors. Brick & mortar stores going online had limited content to provide this new channel, and what data was available was not nearly sufficient to replace the tactile connection shoppers had to products in a store.

Some vendors, such as B&H invested heavily in this area and it proved to be a strong differentiator for them.

Basic E-Commerce content (which I describe as features, specs, description, and images) for areas such as electronics has become a commodity – with companies like CNET, Etlize able to provide this data in a structured format, cheaply and easily. Others like Sellpoint provide more in-depth, multimedia content for products, raising the bar even higher.

So – does this remove the content based differentiators for companies?

Let’s evaluate what this really means…

Companies able to produce detailed content did it because they had domain experts working for them. People who knew what they were talking about – and knew the products inside and out. They were authentic.

Authenticity is something that retailers, bloggers, content producers, and just about everybody should, and does strive for. Authenticity leads to trust, which in this day and age, is a very valuable commodity.

These companies haven’t lost these people – its just become harder for them to prove their authenticity. There is a sea of vendors that provide similar data online. How does one differentiate themselves?

As mentioned earlier – companies like etilize, sellpoint, cnet etc. have raised the bar on what data can be purchased by vendors. Companies like B&H now need more than just the basic data to be able to utilize it as a differentiator. They now need to provide more and, better data, in more formats to allow customers to consume it any way they desire, whenever, and wherever they want.

The challenge here isn’t just limited to the management of the data created by the in house team. It extends to the correct classification, structure, and storage of data that vendors are gathering from third parties – like the manufactureres themselves, companies like cnet, etilize and sellpoint, as well as User Generated Content like reviews or ratings.  It also means that they have to be able to provide this data for any output format desired – whether they are catalogs, brochures, the website, a mobile campaign, etc.

The internal content creation itself can present a challenge. Ideally, all the information that rests with each individual Domain Expert should be frictionlessly shared among all of them. The customer experience should be the same no matter what channel they choose to interact over. This requires that all the data should be available in all the channels.

Getting everybody to share their data in a coherent, structured format – is more of a challenge than it sounds. Even little things like formatting, spellings etc might trip up systems around guided navigation etc. (example a search engine won’t see Black, and blk as the same thing, when creating navigation options ). Creating the templates, with the appropriate validation, error checking, formatting, and structure for your domain experts to use is another piece of the challenge.

As you can see – “data” can be a hairy issue. Especially product data that needs to be uniform, clean and structured, for purposes such as search, browse, print, catalogs, ads, and marketing initiatives.

There are some solutions to the problem. Technologies such as Produt Information Management systems are reaching a level of maturity that allows for unprecedented control and flexibility over content.  (PIM’s are not the same as a CMS, but that’s a post for another day).

There are many different approaches to the solution – every company (Fulltilt, SteboSystems, Hybris etc.) have approached it differently, but the end goal is very similar. Clean, managemeable, structured, data that can be imported from, and exported to,  almost any location in any format, and style.

If any of this resonates with you, or if you have any experience in this – please let me know if you think I’ve missed something, or if there’s anything that should be added to the challenges faced during the creation and maintenance of content.

In the next post I’ll briefly discuss what I think can be achieved if retailers and manufacturers had the ability to create, maintain, and distribute data, in a clean, standardized format.