If you’re President Obama, say, these figures look pretty good:

If you’re the Prime Minister of England, not so much:

This is called sentiment analysis. Essentially what we’re doing is scanning the text of recent tweets referencing a website for positive or negative phrases, like “I love this” or “this sucks”. In reality the rules are a lot more complicated, with some expressions being strongly opinionated, and others being contradictory. We also have to account for the millions of ways you can link to a site, including services like bit.ly.

We’ve just rolled this feature and a few others out for you to play with, including:

• Much better testing of Feeds. We now support more formats (like Atom), analyse the frequency of your posts and score more intelligently.
• Visual explanation of popularity changes. See how popular a site is and how its popularity is changing.
• Badges containing your score, for you to put on your website (scroll to the bottom of a report). These are updated automatically as your score changes.

Social media testing, including sentiment analysis will be featuring in the forthcoming SiteRay 4.

Try this out on your own website – as always we’d love to hear your feedback.

We’ve just made a number of key changes to Nibbler.

• Reports now have Summary scores, covering Accessibility, Marketing, Experience and Technology.
• We’ve simplified how reports are displayed, hiding most of the fiddly detail unless you request it.
• Scoring for all tests is now simpler and more consistent.

You can now see, for instance, that some sites are built really well but not as effectively marketed. Or vice versa.

We still have a long way to go of course. The scoring is very rough at the moment and there are lot of holes – things we don’t test for that will make the results far more representative overall (for example: Facebook – the 2^nd most popular website in the world – currently gets a 7.5 for marketing).

Future updates will bring more tests, and a lot of refinement to what we already have. Once the scores are reliable enough, we’ll be displaying a league table of the best sites.

Here’s a colour analysis on one of our own webpages: Nibbler.

On the left is the perceived colour palette, i.e. the colours people notice. This is similar to the most commonly used colours, but with consideration for edges, gradients and other tricks of human perception. For now, you might think of it as ‘magic’.

Now compare this with two other websites, first Apple. Notice how few colours Apple use, and how consistently they are applied:

And the self-proclaimed Worst-Website-In-The-World, which is a little less restrained:

Even by only looking at the perceived palette (i.e. no understanding of the page itself), certain patterns emerge:

• Tasteful designs tend to have a high bias of desaturated colours (greys, muted pastels etc) with infrequent use of stronger colour.
• Hues are usually concentrated around a finite number of bands, and these often compliment.
• A single colour usually dominates (think ‘white space’)
• Messy designs have broader, haphazard palettes. Multiple, highly saturated colours are nearly always hideous.

Of course I’m merely testing against my own tastes here, and this crude analysis has no appreciation of context. Surely, someone could demonstrate a beautiful page that violates all of the above generalisations? Probably.

The next steps should be particularly interesting. I’ll be experimentally extending the analysis to perceive some other design ’standards’:

• Complexity
• Shapes (particularly angular versus curved)
• Uniformity of layout
• Possible partitioning of text / layout / image regions (e.g. “this part is an image and so should be analysis differently”)

Ultimately the results from all of these could be put into a machine learning system, which would ask users to rate on sites, and attempt to ‘learn’ what factors most contribute to predicting their response.

Given that “ugly” and “beautiful” are so subjective, this is likely to score for less subjective criteria, like “tastefulness”,  “warmth” and  “boldness”. Who knows where this might lead. Could a future SiteRay comment on the design of your website?

Search engines change their behaviour constantly, and it is always kept secret, so the best answer anyone can give involves a degree of research and guesswork. SEO researchers constantly try different strategies and notice successful patterns in what does or doesn’t work, as do we.

The CSS code itself appears to be ignored by search engines, and there’s not much apparent benefit to them looking at it; the whole point of CSS is that it adds presentation to the content of your pages, not content itself. (One exception: detecting black hat CSS techniques, like keyword stuffing in invisible regions).

However, using CSS properly does enable your pages to be smaller and load faster, which has some positive effect on SEO. It also allows the code in your pages to be ordered in a way that can emphasise certain content, allowing potentially better SEO. For example, using CSS you can present the content for your pages higher in your code (as Google sees it) than say your navigation.

CSS is one technology to use as part of an SEO strategy, but it’s no panacea. Of course, CSS is great for lots of other reasons – like accessibility, printability, maintainability -  and you’d be hard pressed to make a modern site without it now, so our recommendation is simple: use CSS, and use it properly.

A week after going live, we had 27,000 links pointing to us (mostly blogs), a year later over 1.8 million. Our site became more popular than the University of Nottingham in under 6 months, and that’s with zero offline marketing and five employees.

As popularity rose we were featured on media throughout the globe, from Canadian TV to Spanish radio. We had enquiries from FORTUNE companies about a commercial version, and it didn’t take long before we wrote one.

Unfortunately, the original free Sitescore was just a humble script we’d cobbled together in a week or two. It wasn’t designed to handle that much traffic, in fact, we didn’t even bother using analytics on it at first.  We were also a much smaller company back then, and Sitescore’s popularity was costing us more than we could afford, in time and hosting costs. So about a year after it launched, we had to shut it down.

We released another free Sitescore shortly (“Sitescore 2”) after, and unfortunately we got it wrong. We based it on the engine from our commercial version which was much better, but sadly not scalable enough for the general public. The same problem occurred, and we had to return to our drawing boards.

Over two years passed since the original Sitescore was shut down, and we’ve grown a lot. We had to rename Sitescore to SiteRay due to a trademark conflict (the name is too similar to Sitecore, a renowned CMS company). SiteRay is now on version 3 and we have over 400 paying customers.

And so it came time to write a new, free Sitescore. We called it Nibbler (“take a taste of your website”).

We had learnt the hard way not to underestimate demand, so we redesigned everything from the ground up to handle huge volumes of traffic which could ramp up quickly. We decided to base Nibbler on Amazon’s Cloud technology, which means we don’t even need our own physical servers – we just rent computing power from Amazon. Using the cloud means we increase our capacity at any time, automatically, and then reduce it again when demand subsides. Previously we’d have to wake up in the night and start shopping around for biggest, faster servers.

Nibbler is only in alpha right now, which means there’s loads of broken or incomplete stuff we know about, but we think some people will still find it useful. We have a team working on developing it constantly, and there’s a lot planned over the coming months.

Take a look and let us know what you think. We’ll be keeping this blog updated with stories of our new adventures as they happen.

The criteria for any test – not just ours – are inherently arbitrary. You might be using W3C’s validator, for instance, and assume that because it’s got W3C stamped over it, that it’s the perfect way of testing that your HTML is ‘correct’.

And you would be wrong.

Try implementing a secure e-commerce page that passes W3C validation. You’ll be wanting to turn off autocomplete of course, because you can’t have credit card numbers and the like being stored in the user’s browser. Unfortunately, the HTML for that – which works in every browser – will cause you to fail W3C.

Of course W3C has a get-out clause: they’re only testing what they claim to be testing, which is conformance against the HTML spec. Great news for them, bunk-all good to those trying to use their test in the real world.

This one problem could be fixed by simply adding autocomplete as an officially supported feature – but that wouldn’t change the fundamental issue: somewhere, someone is deciding what these tests should cover, and you won’t always agree. Even when it’s us.

This problem is unavoidable. The test-designer can argue their case, use extensive research and a thorough testing methodology, but fundamentally you’re making a choice to accept their conclusions and ascribe them a certain level of worth. How much worth is up to you.

For our part, we’re trying to avoid the avoidable issues like W3C. That means regularly updating, being as open as possible, and involving our community in the refinement of our tests. We believe that by doing this consistently over a long period of time, we can earn the trust of users.

But remember, no test is perfect.

Computers are really good at following precise instructions very quickly, which is pretty much the opposite of us. If you can explain the problem in simple steps, then a computer is your … erm … man:

1. Find all images on a page
2. Find out what size they claim to be
3. Find out their actual size
4. If 2 is different from 3, report an error

A rickety old PC could run a test like this with contemptuous ease, and it would always uncover images with the wrong sizes a billion times faster than you could. These tests are super easy to write, and reliable to run.

Now consider spell checking. This simple set of rules should do the trick:

1. Find all words on a page (split them by spaces and punctuation marks).
2. Look each word up in a dictionary.
3. If a word is not found, flag it as mis-spelt.

Except this is completely wrong. For starters, look at these ‘words’:

3.14157

30^th

BBC

£23,000

2/07/1979

bob@example.com

www.example.com

All of these won’t be found in a dictionary. There are certain rules which say that numbers are not words, and that uppercase words are acronyms (in English at least). Ordinals (like 1^st, 2^nd, 3^rd) go on forever, and you can’t add each one to the dictionary. Instead, you need a tonne of custom rules to recognise these exceptions, and ignore them correctly.

Don’t forget that spelling can vary depending on context (e.g. the first letter of a new sentence should be uppercase, but proper nouns like ‘Dave’ should always be uppercase). Oh, and you want to check spelling in languages other than English, right? Good luck with that.

Of course we have spell checkers in Word, and now in our web browsers, and they’re pretty good too. So although this problem is a total bitch, it is possible. And this gives us hope for the other bitchy problems we have our hearts set on solving.

Note however, that automated spell checkers are definitely not perfect. And unless they become actually as smart as human beings, they never will be. Try this in Word:

“This is what the student wrote: I dont no how to spel”.

Now as a human it’s obvious that the spelling ‘mistakes’ here were intentional, and in fact essential to the meaning. But without a near human level of understanding of this sentence, no computer is going to know that. Nor is it worth making our spell checkers 8 billion times more complicated to try and eliminate the 0.001% of fringe cases where they fail, if we even knew how to do that, which we don’t (yet).

And so it is with testing websites.

There are ‘soft’ tests, for easy problems – like counting broken links. These will have extremely high accuracy, but tend to be relatively superficial. They form the majority of automated tests available today.

And there are ‘hard’ tests, for problems like spell checking, or identifying different designs in your site, or telling you your graphic design makes my eyes bleed. These are vastly more complex, imperfect sets of rules which approximate things our human brains can often do easily. With work, they can be right over 99% of the time, but they never be perfect. Just like the spell checker in Word though, they can still be very useful.

We have a lot of interest in the hard problems, which is where we believe the most valuable feedback on a website can be identified. Wouldn’t it be great if SiteRay could tell you, for example, that the strategy for your website was wrong? Maybe, one day, it will…

We love the web. We want to make it a better place. Our challenge is figuring out how to do that best.

All our products revolve around the simple goal of making websites better, which can broken into 3:

• We can identify what can be improved with a website.
• We can recommend what can be done about it.
• We can change the website.

Not everything we do will accomplish all of this at once. SiteRay and Nibbler – our only two products for now – focus exclusively on the first two. We’re going to get amazingly great at those before we worry about helping with changes: that’s a whole new product in itself.

As an example, this is how an ideal SiteRay might work:

• SiteRay looks at a website.
• SiteRay asks select questions about this website, e.g. “what are your key actions?”
• Issues are identified, such as “the actions in this site are displayed very low in the page, below the fold”.
• Recommendations are made, such as “change the design to bring actions higher up the page (see these examples)”.

This example neatly highlights the type of problem which computers are not very good at identifying, and that is one of the reasons our jobs are so-incredibly-cool, because we get to solve it.

How does that pay the bills?

I’m a firm believer that companies founded solely to make money, and without a greater reason for being, usually die with neither.

Our aim is to make websites better, and fortunately there are a lot of people who are willing to pay for if  they’re convinced we can deliver it. There is no doubting the size, growth or significance of this market, and it isn’t likely to be going anywhere soon.

For sure, there are some private Excel-plus-back-of-napkin calculations here, but fundamentally: as long as we can deliver what we’re promising, money isn’t likely to be a problem.

Delivery

For the foreseeable, we’ll deliver all of our solutions as SAAS (software as a service) – i.e. we provide a complete solution, hosted and supported, in exchange for relatively small regular payments. Most of this service will be provided via the cloud.

This is why:

1. Delivery scales. We can share resources such as servers between many clients, offering a lower cost to them than they could source themselves.
2. Conversely, non-SAAS does not scale. For each customer who installs on their own hardware, we need significantly more resources to support them, resulting in higher costs for them and us. We grow slower.
3. Customers only pay for what they use, as do we (when we use the cloud). Economists love this sort of thing: the efficiency allows us to keep prices low and keep our finances healthy, even during a downturn.
4. Low commitment is a strong incentive for customers to try us.
5. Regular payments allow us to give all customers ‘the latest version’, without big charges for annual upgrades, and different customers using different versions of our products. This means less support for us, which means lower costs for them.

Basically everyone wins, except for those who really want to install on their own servers. We have a few of those customers who we’ll continue to support, but we’re not planning on adding any more of them.

Summary

Write software that helps people make their websites better. Don’t shy away from the hard problems. Deliver it as SAAS. And let the finances take care of themselves (more or less).

Nibbler is a free, cut-down version of SiteRay, but designed to be useful and fun by itself. It takes a 5 page sample of any website and rates it for a tonne of criteria, covering accessibility, SEO and social networking among others. We even tell you how dated your website looks: fear the words “your website is sooo 1995”.

We’re still in alpha right now, which means stuff may break, and we know it. In our defence, if you’re not embarrassed by the first version of your product, you’ve launched too late. If you find something, please let us know via the Feedback tab.

Nibbler is the long term descendant of Sitescore, a free tool we released many years ago but couldn’t keep online due to immense demand. Since then our technology has got a lot better – Nibbler has been designed to run on the cloud and should cope far better with the squillions of users we hope will one day be using it.

This wouldn’t have been possible without a wonderful team of hyper-motivated and super-smart individuals, with particular thanks to Greg Heafield, Jake Noble, Andy Waite, Tim Unwin, Alison Springall, Ben Roberts and everyone who supported them. Thanks guys, you’re awesome.

We love how you hold the entire knowledge of the world, yet keep on growing.

We love how you accept our every question, and answer them in an instant. How you see every opinion on every topic, yet choose to judge no one.

We love the way that your trillions of parts somehow just work, and keep working, despite a world of attacks and faulty parts. Despite it all, you keep on getting better, faster, ceaselessly, and seemingly without limit.

We love that we can see you at home, at work or even outdoors, and – for the most part – not even have to think about you costing us anything. You give us the world, and ask for nothing in return.

Because of you, almost everything is cheaper, and easier to find, and buy. Millions now have access to freedoms and information dreamt impossible only 10 years ago. Oppressive regimes cannot escape your reach, a true champion of the people.

You make us laugh, you make us cry. You show us with brutal honesty our dark side.

And of course without you, web peeps like us would be out of a cushy job.

Forever yours,

The Silktide Team