ThumbWhere

ql.io is interesting.

2011-12-02T20:11:00.018+11:00

ql.io provides the same function as the ThumbWhere web service composition layer and was written to solve the same problem with the same philosophy of putting more control in the hands of developers.

Where it differs is that like YQL it is read-only so you can't compose everything.

What is a Web Service Composition Layer?

The aim is to push the back-and-forth traffic closer to the services and to avoid the return trip latency.

As developers one big constraint is the speed of light. To send a request, get the result, send another request based on the result, all this back and forth can add many seconds to the start up time of an application.

We can't make light go any faster, but we can make the total trip shorter.

If you need to make 6 calls to different systems to complete authentication, catalogue, entitlements and analytics, you are adding a lot to your client applications start up time.

Another aim to make sure that developers are not overly constrained by workflows imposed by services. You can be more granular with your services and know that the client can compose the workflow that suits the application or platform they are using. eg. PC vs Mobile can have radically different workflows for the same services.

As a service designer, you don't need to second guess every case that developers may come up with.
As a developer and service consumer, you can construct a perfect single web service call, every time.

ql.io combines JSON and SQL

I can appreciate that it is compact, but i'm concerned that it's hard to make existing web services look like tables in a database, and that for many front end developers SQL would be a new skill they would have to acquire,.

ThumbWhere's implementaion combines JavaScript and XPATH. This was carefully chosen to take advantage of the native XPATH engines that are available on the server side and in modern web browsers, XML/XPATH is at least a familiar technology to most front end developers.

In the environment where ThumbWhere was developed JavaScript and XSLT/XPATH was and is still a heavily used language for client side development, this was a consideration of the design.

(Here is the Composition layer being called from JavaScript).

Here it is being called from C#

Some features in ThumbWhere has that it's not obvious are in ql.io (site is down at the time of writing this article).

Asynchronous call backs: As the result streamed back from the server multiple call-backs can be triggered.

Generic feed integration (you can pull in any XML feed or existing REST web service call and manipulate it, use content as parameters for other calls in the composition).

Flow control (if/while) that allows a call to for example make decisions based on the return values of other.

Response Mutation: ThumbWhere lets you move, rename and delete elements of the feed or

API responses before they are sent back to the client.

Request Parallelisation: ThumbWhere can detect or lets you mark requests that can be run

in parallel.

Write & Execute: ql.io only has has a select statement (at this point). It's read only. Thumbwhere allows you to compose read and write operations.

What ql.io does provide is a wonderful join syntax and ability to perform complex reductions based on relations - which can be done using XPATH in ThumbWhere - but is not as nice as JOIN :)

If anything this is vindication for our approach.

Its a bit hard to try and induce a new way of thinking so I'm really glad that ql.io has put a stake in the ground for saner web services. With ql.io and YQL out there now, my elevator pitch will get less blank looks :)

I'm going to strongly consider rolling this into our notification/feed/edge stack as it's such a close fit with what we already have.

First release of 1.1 is Live!

2011-11-23T12:07:00.022+11:00

Greetings ThumbWhere clients and spectators.

Over the last 2 weeks we have rolled out some new services as well as a whole new API layer to support one of our longest running trial members.

The old public facing API Twitter integration demo site has been switched off. It was using the legacy (v0.0) API and it needs to be rewritten to use our latest JavaScript Service library. For now it redirects to this blog. Of note I believe it was the first tool ever that let you upload audio, images and video to Twitter from your mobile phone. Fun times :)

We will eventually replace it with our newly planned service sign-up/administration portal.

That's right folks, we are slowly on the way to going public and allowing anyone to sign up and start using our services.

Over the last 4 years we have run 8 campaigns with 6 different organisations to validate and test our product.

Just these word of mouth campaigns and other IPTV development has been keeping us busy for the last year - but we have continued to plod on with our ThumbWhere product development.

This most recent update (v1.1) represents a lot of the learnings and results of R&D from the last year.

Notable changes:

The v0.0 and v1.0 API's were versioned, the v1.1 API is versioned and modular using SOA principles.
FTPS and FTPES are now supported for first line storage as well as with CDN storage with Amazon S3 and more to come.
Campaign members can now email media and have it ingested into ThumbWhere with their own email address or a pre-generated ThumbWhere specific email address.
We have changes under the hood to improve the reporting of ingestion and transcoding workflow exceptions and new operations in the primary distributed workflow engine.
Auto-generated Web Service interface libraries for .NET and JavaScript. PHP and Java are on the way.
Auto-generated windows tools . PowerShell 'Snap-ins' for each of the new services to allow easy scripting and tool writing. Windows workflow 'Activities' are on the way.
Media can be FTPd directly to ThumbWhere on a per member or campaign basis.
A lot of work has gone into making our internal development process more agile with a lot more code generation. Now we can design, generate and stand-up a new SOA service and supporting repositories in a matter of hours.

Over the next month expect to see a lot more news about the latest v1.1 Content and Social APIs

Cheers.

SFTP Added To ThumbWhere Storage

2010-08-22T07:55:00.002+10:00

Checked in SFTP support.

We need to test with more SFTP servers than we have in the development environment, but this is all ready for testing.

Now for FTPS and plain old FTP.

One side effect of the AmazonS3 Storage integration is that now every time we run automated my integration tests it costs us a few fractions of a cent. :)

But it is well worth it :)

ThumbWhere AmazonS3 Support

2010-07-04T17:55:00.003+10:00

I just checked in the test candidate for external storage which supports Amazon S3 and FTP.

When this is deployed, API licensees can provide one or more Amazon S3 or FTP accounts and ThumbWhere will use that to store your media.

You specify an account and then a number of selectors which are used to select media for inclusion (or exclusion) with respect to a designated storage provider.

For example. You might want your videos stored in one S3 account and your thumbnails in another. In fact you might want to deploy certain thumbnails to a different type of storage altogether.

So by configuration of selectors and storage endpoints you can obtain a high degree of control over where your content is actually stored.

The system is architected to support generic storage providers via a plug-in model so I’m now investigating what others I can support in the first release. If I don’t see any easy wins then S3 and FTP will be the only ones in the first release.

I spent a good portion of Sunday writing automated tests to ensure that the core flow processor was able to handle the cases of first time deploy, revocation and update with the minimum amount of API calls and traffic. If multiple deploy instructions for the same media gets into the queue the engine is able to avoid double handling. If some of the media is modified and re-transcoded but all the media is marked for redeployment, only the media that has actually changed will be deployed.

You put a bit more thought into these things when you get a running bill from Amazon per API call and byte shipped. This weekends efforts added up to 3c :)

I’m also considering personalised storage, so actual end users of the system (members of each social network) could in theory provide their own S3 account info and have their own media shipped off to their own storage.

The media in external storage is represented by extra URL elements in the raw feed XML. The default XSLT for the account specific feeds will make this transparent by selecting a single URL and favouring external storage over files hosted on internal ThumbWhere storage.

We will start to factor this into our pricing model but the immediate effect is that this will allow you to ship most of your media traffic to cheaper storage and it will allow us to control our own storage costs.

Update

2010-07-02T12:19:00.004+10:00

We've been rather busy for the last year and a half working in the emerging interactive television market so unfortunately there has been no time or resources to devote to the public API as I've spent most of my time either head down coding or out of the country working with some amazing organisations.

The workload has recently started to get reasonable so I'm now looking again at deploying updates to the white site social networking site to take advantage of the last year or so of work in the commercial API.

This means that finally some of the features requested in http://feedback.thumbwhere.com that have already been implemented in the API will start to see the light of day.

Also we are taking on some pro-bono clients who have expressed an interest in using the API to make the world a batter and more interesting place. These will act as example implementations for integrating with the API. At the moment as we have been doing all of the integration, public facing documentation is a little sparse. This exercise is an attempt to do some good and get some good example code. The long term aim is to start opening up the API to a general developer community.

It's amazing to see what people have been building with the API.

Thanks everyone for your ongoing support and encouragement.

ThumbWhere Image upload now supported by MahTweets

2009-09-06T10:45:00.001+10:00

“MahTweets is a rich, awesome, twitter client, feature inline media, filtering, tracking (saved searches) and much more”

MahTweets can be downloaded from http://theleagueofpaul.com/mahtweets/

The ThumbWhere plugin was developed by @WillHughes

It supports Image uploading and uses ThumbWhere’s URL shrinking service ( tny.tw ).

You can get a ThumbWhere account at http://thumbwhere.com

Also you don’t have an account you can always MMS images, video and audio to +61-447-100-293 and ThumbWhere will publish it anonymously to the public feed.

ThumbWhere Launched

2009-04-29T12:30:00.002+10:00

For some reason blogger had a massive password fail with my google account linked to this blog. That now seems to have been resolved.

ThumbWhere launched last month - 1 day early :)

Getting heaps of submissions and suggestions in http://feedback.thumbwhere.com

First update will be submission via email to allow all the iPhone users to use ThumbWhere.

ThumbWhere Is On Twitter

2009-03-14T02:04:00.003+11:00

Almost there. There are no pending bugs.

The 'official' soft launch is in 11 days - but the site will be operational well before then.

The site is a 'white site' / 'wire-frame' so its ugly - but very very functional.

At the moment its only integrating with Twitter - but new services will be added very quickly. I have plugins for a few other social networks (not naming any names yet) already written. If there is one you want to prioritise - click on Vote For Features and request it.

I'm targeting older MMS based phones for the first month so users with non MMS phones that require email will need to wait a little longer.

If you have recently joined Twitter and your phone is a plain old mobile phone that can MMS then ThumbWhere is ideal for you if you want to send photos, videos or sound recordings when you are on the go.

Did I just see a flurry of activity on the ThumbWhere Twitter account?

Testing Continues

2009-03-02T21:36:00.002+11:00

The bugs continue to be found and squashed.

Also making some minor enhancements and planning for some new features.

First Posts Through ThumbWhere Public Front End

2009-02-26T22:57:00.003+11:00

The public front end transitioned from development to staging server today. First tests all good with only a few configuration issues to deal with.

Sent an MMS and SMS to Twitter with no issues.

Will start handing out the test URL tomorrow to a select few.

Testing 1 2 3...

2009-02-12T09:41:00.006+11:00

Completed first end-to-end test with the new front-end driven by my state-machine library by posting a pic to the @ThumbWhereDEV Twitter account.

I had the core ThumbWhere engine up and running late 2007, and its taken a long time to get the funds and the time together to complete the system. This has been a case of 'launching real soon now' for the last 12 months.

ThumbWhere's core business is leasing out its social media API to other companies. The good side is that excluding my time, ThumbWhere's has been self-funding for the 6 last months and has another 6 months covered and more clients lining up. So many in fact, this is one of the reasons the public facing site has been such a long time coming. I've been loathe to risk downtime just to support a public free version of the services. I have a suite of unit tests and an automated upgrade and rollback process, still I'm loath to potentially disrupt customers.

Call me, cautious.

The major usage of the service has been the Competition API. This allows anyone to create an online competition (example) of multiple rounds where media can be uploaded, rated, tagged, commented on and voted on. Customers make API calls and pull XML feeds back. ThumbWhere provides a simple mechanism to customise and produce your own feeds.

This period of profitable abstention has given me time to make improvements and develop my own AJAX state machine MVC like framework. This was a large initial investment in time, but now allows me to very rapidly prototype and add features.

Luckily at the end of Feb I have a window of a few weeks between major client campaigns which will allow me to deploy the latest enhancements.

This will mean there will be a public service based on the API that will allow you to SMS/MMS or upload text, images, video and audio and have it posted to the social network of your choice. I'm starting with Twitter and other social network's will follow as I configure them in.

Considering adding either TwitPic, BrightKite or FaceBook as the next.

TwitPic would be an interesting choice because the Twitter App effectively replicates the same features of TwitPic and then some. It would allow people to post video to TwitPic if I include the a link to the media on ThumbWhere. Considering.

I'm now pairing the app back to basic unstyled HTML and will launch with that. Planning to add my skinning moduleover the top to control the presentation. In effect this will be a demo/proof of concept for some advanced technologies I've been working on for the last year.

ThumbWhere Has A Blog

2009-02-06T11:39:00.000+11:00

Now all I need to do is launch the site :)

QR Code support

2008-07-04T14:56:00.002+10:00

I have put this up from the old HTML media item pages to make things a bit cleaner.

All media submitted to ThumbWhere has an associated QR Code. If you scan the barcode using QR Code reading software on your mobile phone, a version of the media compatible with your phone will be loaded onto your phone.

Read about QR Codes.

If you need QR code reader for your phone, here are two I have tested already.

QuickMark
This requires registration but they support a large number of phones. You will need to install from your PC.
Scanning style is Frame, Hope and Click - so its not automatic and can take a few goes.
Tested fine on my WM6 JasJam.
I-Nigma
Enter http://i-nigma.mobi into your mobile phone's web browser.
This tracks scanned QR codes via a redirect. So whatever URL you scan in, i-nigma will know about it.
Scanning style is Automatic where the phone security model allows for it - as soon as it recognises the QR code, it will decode it.
It works well on my WM6 JasJam (Automatic), but completely crashes my Nokia 6280 (Manual).

Tips And Trcks

2008-02-05T14:58:00.000+11:00

Save the ThumbWhere MMS Number

When you see something you want to share with your friends, don't fumble with your phone - be prepared. Save 61-447-100-293 in your phone contacts under "AAA ThumbWhere" so you can quickly send photos, video and audio to ThumbWhere when an opportunity presents itself.

Send a Description

When you send to ThumbWhere, put the title in the message subject or body. Pictures with text descriptions are automatically tagged and are more easily found.

If you send in a blank subject and put text in the body, the body will be treated as the subject.

Rotate Your Pictures

You can add special commands to the start of the subject to rotate your photos. This is useful if your phone's camera functions don't autmatically rotate an image before sending it via MMS

> Will rotate the photo right (clockwise)

So if you sent in the following subject

>I Love My Kitty!

Your picture will be rotated to the right and given the subject of "I Love My Kitty."

Caption Your Photos LOLcat Style!

This blog post has a more in depth discussion of the codes used to generate captions.

Any text you put at the start of the subject in square brackets

[Like This]

will be turned into a caption. The remainder of the subject will be left as is, so if you send

[Fear Me Human]Why is my kitty growling?

It will caption the image with "Fear Me Human" and the subect will be "Why is my kitty growling?"

A Caption can be positioned in three rows on your photo, [Top/Middle/Bottom]. The / character is used to switch to the next row.

A more complicated example is

[I have a human/It feeds me/I may let it live]My Kitty Loves Her Food!

which will caption the image on three lines "I Have A Human" will be at the top, "It Feeds Me" will be in the middle and "I May Let It Live" will be at the top. The Subject of the message will be "My Kitty Loves Her Food!"

You can of couse leave the first two rows blank if you want your caption at the bottom as in this next example,

[//Die Human!]Why does my Kitty Have Glowing Eyes?

which will caption the image at the bottom with "Die Human!". The Subject of the message will be "Why does my Kitty Have Glowing Eyes?"

Putting a <> in your caption will allign the text to the left or the right.

[Chicken!]

will put a caption at the top, left justified and at the bottom right justified.

All captions are automatically capitalised, they just seem to look better that way.

You don't have to end your caption instruction with a ] if you are not going put some text after it, such that

[nom nom nom

will still work as a top caption even though the caption command is missing its final ]

If you run out of room in the Subject, keep the Subject blank and use the message body. If the Subject is blank, the Body will be treated as the subject and image commands such as caption and rotation will come from the body.

If you think these instruction can be made better - please send me an email.

First Light

2007-11-20T13:00:00.002+11:00

The ThumbWhere API is being used over XMAS by a few people to record their holiday snaps and add them to Twitter and Facebook.

Fun times!