Can you smell the truth? Singed flesh and Local Gov web site management.

FWIW, I ran a District LG (Local Government) website for 10 years.  I developed a CMS as I went along, html then .asp then shortly after in php.

About two years in I realised there were clearly two types of content.

  • Static
  • Dynamic

I also realised there were broadly two types of content owners

  • web-hostile
  • web-friendly

So the CMS I made, and therefore the website, 'evolved' to suit these facts of (LG) life.

For the most part I left the static content as html, and only I edited it - I just made a simple "request for change" online form so that the web-hostile crowd :

a) identified the url
b) identified the paragraph they wanted changing
c) did most of the typing for me.

At every turn I would run up and show them all the dynamic content they COULD be playing with, with minor successes.

But by definition "static content" hardly ever changed - if it did change more than say, once inside a month, it was time to have a meeting and dicuss the real message that the owner wanted to get out to the public and by and large those conversations lead to me discovering that they needed something more, usually based on some kind of datastore CRUD element or application.

Taking this course also meant that the static content pages could contain as much complex information as was necessary, I was free of any CMS constraints about layout or content, bringing in webservices from off the web for graphing - linking to databases from .xls files etc.

The web-friendly crowd, who were writing and maintaining the really interesting stuff had access to and used more KM-like content such as dynamic FAQs, articles, events, image libraries  etc.  Some of this dynamic stuff was piped onto the home page partly as "reward" for taking the time to contribute, and was a very effective motivator.

There were a few web-friendly people who only had static files, but that could be very rich in both form and content because they were willing to work with me to develop the applications sitting beneath the content - custom applications made to solve their particular needs e.g. queryable Air quality tables with and historical data with graphs and maps, complex election reporting etc.

One of the biggest benefits in developing this system was this LG fact of life: The web-friendly users (call them "bright young things") never stayed in the job for more than about a year, then they got promoted, moved sideways in LG or just plain quit.

This LG had a policy of not replacing people straight away, mostly for budgetary reasons (I don't know how widespread this practice is).

So we can add to the mix of LG facts of life;

  • People quit and are not replaced immediately
  • Some people have several roles in the organisation

So if the boss of the web-friendly "bright young thing" was not able to confirm that the dynamic content was still valid, I could rip the lot off the site with one action (or hide it, more accurately).

Also, it occassionally happened that a web-friendly person was replaced with a web-hostile person, and the same scenario played out.  Articles just ran out of time and got removed automatically anyhow, other dynamic content was ripped off by consent.

I had distinct problems with the web-hostile crowd in that almost to a (wo)man they sent me content which was unsuitable for publication on a website "as is".  Full of officious complicated text, packed with jargon and phrases such as "the council will ..." not written in plain english, nighmarish.

I would sit for hours explaining how I intended changing their copy to make it web friendly (my favourite lead comment was "can you explain what this document means?" - they would say something in plain english and I would type it out quickly.  That became the lead paragraph.) and sometimes I would be told "just put it on as it is", which I rarely did except in the case of docs which clearly had some legal connotations.

Anyhow that was the scenario, I am sure you recognise some patterns there (or anti-patterns).

This system worked well.  The site featured in the annual Socitm "Better Connected" top 20 websites 4 times in that 10 year period, and was usually in the top 10 percentile in Sitemorse except when I was indulging in some DTD-swopping for one reason or another.

I did some hunting around and it is possible to score a piece of text on its readability, by working out the length of words, words per sentence etc. I planned to hardwire this into the CMS and score the static content and rank the worst pages, and also to work as a 'nag' to all online content as it was being saved. They ended my contract before I could implement it.

My points are these:

  1. You will always need a gatekeeper, and make sure they have the necessary clout (but they will be unpopular)
  2. Web-hostile people should be kept away from the CMS, or at least the publish button
  3. Software alone cannot guarantee you will carry good content, but it can have an important role to play
  4. LG CMSs are not designed from the ground up for local government, and it shows
  5. LG CMSs stifle innovation almost as much as does carrot-chasing Socitm scores

Poor management skills elsewhere in the organisation, such as the inability to delegate, will cause inapropriate content to surface on your website - but software (such as layers of access control) is not the answer to that problem - it just throws up another level of complexity to confound users.

Tools are emerging designed to help people to create better content but until your CMS provider adds the algorithms/applications you are stuck.

I originally posted this as a reply to another blogger - the blog name escapes me now, I am reproducing it here now as an aide-memoire really, plus I am looking for employment in Local Gov in the UK again.

Why we need to unlock Charities data now.

I just learned that The Charities Commission are refusing to properly open their data.

This is this very bad news for the public and you should immediately add your vote to have this data unlocked.

Can you just go and do it now, before you forget.

That's the short version, the long version follows, and right at the end is another link to the same unlocking service for you to add your voice.

Amidst the public clamour to put out council spending data, and the mutterings and dealings about taxonomies and controlled lists - is the realisation that all this information is a lot less useful if you cannot de-reference any of it. 

Take this simple example.

Payment: £1,500
Date: 1 Apr 2010
To: Company A

Self-describing data can be de-referenced, so imagine that record could have read more like:

Payment-period:  2010-01-01 - 2010-03-31
Payment-date: 2010-04-01
Payment-amount: 1,500.00 UKP
To: companieshouse .website.com /Company_A_Ltd_details
Alternative URL companieshouse .website.com/123456789

As you can see in my very much simplified example the former recipient Company A is none of the following entities:

  • Company A PLC
  • Company A Holdings
  • Company A (Jersey) LTD
  • Company A and sons
  • Company A
But can be de-referenced to a single unambiguous entity for which I can train a machine to fetch data from and display that data ALONG WITH the payment details.

Ok, so what does this have to do with Charities?

Because Charities are very big recipients of public monies in the form of grants, those grants appear as payments.

The handling of grants within Councils can be a really convoluted affair, messy and expensive.

Grants attract other grants and there is a tendency for strata of Local Government and NGOs to snuggle together in order to spread and share risk, something they call "matched funding".

Very often the Town or Parish Council is entrusted with the final say about whether a grant is allocated, often tied up with a nominated Town or Parish Councillor taking on the responsibility for monitoring the ongoing activity of that charity. 

Many smaller local Charities would not receive any local funding if they did not have a Town or Parish Council to administer those grants - it would be too expensive.

Town and Parish Councillors are unpaid, so the public burden is very light once the money is given to them.

But as this public money passes down between the strata, and ends up being given to Town and Parish Councils, as, yes, payments.

So, go back to my Company A example and this time substitute "Charity A", and that too should be a fixed, unambiguous entity shown as a URL.

To: charities. data. website.com /Charity_A_details
Alternative URL charities. data .website.com /987654321

Slight meander ...

If you have ever sat in a meeting where County and District Council staff and/or members have discussed the handling of grants you will know what I mean.

The political horse-trading, posturing and deals that are done can be at once farcical, shocking and bewildering.  Sometimes it all seems to be about fighting for your corner, making sure your departments' name is associated with the grant, augmenting your budget if you can. The decisions that pop out often seem random unless seen in true light of powerful alliances, political stances and creative accounting.

I sometimes tried to imagine the paper trail of this money.  If you ever need to launder a large amount of cash, surely earmarking it for grants would make it untraceable.  My favourite question was always "... but where is this money coming from?" "The Department of x" would invariably be the reply, "Yes, but where do they get it from?" "Well, the GOVERNMENT stupid!" - "but where do THEY ...." ad nauseum. 

They stopped inviting me after a while.

I have no idea how much a £500 grant for the Scout hut roof actually costs the public in administrative costs, but the cost in biscuits alone must be tremendous.

(I remember a meeting once where the first 10 minutes was taken up with discussing which department's budget was going to be charged for the tea and coffee, we were 8 in the room - so scoff ye not)

I have this mental picture of what how this should work.  Its like a simple straight pipe, money in at one end, money out to the recipients at the other, like fresh water.  Instead we have something resembling a broken digestive system, it twists and turns as more and more departments need to rinse their fingers in the stream as it passes. They need to break its fall, let it collect in stagnant pockets for a while, go a bit rank, then pass it on 'cause its "the end of the financial year", or some more money is flowing in so "we need to spend it, like now, before the end of the month!"

"We've got some money we could use for that" a phrase always guaranteed to infuriate me.

The point at last

The administration of grants is an expensive and time-consuming process which could be improved by Government giving directly to many more charities, or at least giving it directly to Town and Parish Councils.

Ultimately that wasted expense is reducing the amount of cash being given to the Charities.

Opening up Charities data as Linked Open Data (LOD) will help shed light upon the administrative process, as charities can be linked to payments (and vice-versa).

Many of the smaller payments will not appear anywhere unless Town and Parish Councils publish their spending data also.

The likes of Adrian Short (Armchair Auditor) and Chris Taggart (OpenlyLocal) are trying to expose and de-reference Local Government payment details by supplier in order to move forward the examples of open data.

This requires both company details and charity details to be de-referenced, by them, and others like them to link to unique and permanent URLs.as I have described above.

We have been promised that companies data will be opened up as a priority - we similarly need charity data to be opened too.

I want to de-reference companies and charities because they are also employers who feature in my JobsGoLocal job vacancy data streams.

I and others are going to need either a local copy of charities data, or a robust end point which serves this data.

Wouldn't it be better if you were able to see the details of the employer - especially if they are a charity - or horrors, if they are claiming to be a charity, as you were browsing the job vacancies?

So, please sign up to get this data released and blog about or retweet this message - hat tip to Tom Szekeres for kicking the idea off.

Can we have our data back please?

Were you expecting to see transparency on local government IT spending some time soon? Where does a large part of that council tax money go and just what does it pay for?  Read on to get the skinny on why the status quo is likely to remain and has already taken steps to shut you out.

By way of scene setting I need to present some things l learned in ten years of working in local government IT, local government IT truths.

Fear of merger

In some councils the biggest fear among top employees and members (councillors) is that their organisation is merged with a neighbour in order to share resources and save money.  In the resulting consolidation, they might lose their job/status/power.

This truth manifests itself when the time comes to select an IT supplier for a product.

The cost of merging two neighbouring councils will depend to a great deal on how quickly, easily and cheaply integration of back office IT can be achieved. If too many of our software packages match that of any single one of our neighbours, then we will be making ourselves an easy candidate for merger.  We must avoid this in order to maintain the status quo.

If all of our Financial, Housing, Content Management, CRM, ERM, GIS, Planning, Knowledge Management (KM), Internal Email and messaging etc etc systems are different from our neighbours then it will cost untold millions to merge us and therefore will not happen.

In fact as long as only one or two the top six or seven applications overlap we will still be able to stymie any attempts from above to merge us with neighbours.

Lets say we work for District-C.  Our northern border is shared with District-B, our southern shared with District-A.

Now the tenders are in for our Financial Software package, and we, District-C have a choice of Supplier-X, Supplier-Y or Supplier-Z's Financial product.

"So while Supplier-X really does have the right CMS product for us, with wonderful drivers and mappers linking to our existing CRM which would result in massive savings - we have to discount them because District-B already uses it.  We already have GIS and Email in common with District-B, so no, we cannot chose them."

"Similarly Supplier-Ys product, while a good fit, for the same reason - too much in common with District-A."

"Supplier-Z. Staff in the demos said they did not like using it, and we've heard bad things about it from other councils too.  Suppliers-Zs product is the most expensive, and we don't have any Language-Z programmers on staff."

Contract awarded to Supplier-Z then, by unanimous vote. The Fear of merger gene takes precedent, see?

Things become more convoluted if your shared eastern border with District-D is in a different County, but hopefully you get the picture. The threat of merger may only surface every 5 years or so, but it is real in the minds of council people.

When this fact of life was explained to me I was awestruck, it was beautiful and horrible at the same time. I was wide-eyed, I felt like Yossarian in Catch-22. I imagined the country as huge 3D chequerboard, separated by layers of intentional obfuscation which would take absolutely decades to unpick.

Charging yourself up with this knowledge will explain the pre-ordained doom waiting in lie for suggestions for projects like a "National CMS".

Any of this ring true for you, Miss local government officer? Ever overheard those remarks after visits to a neighbouring council "Christ, their systems are easy to use, cheaper too.  Why didn't we buy that?"

Welcome to the overriding logic behind many unfathomable decisions regarding big ticket public IT spends, I mean think 6 or 7 figure sums and multiply it by say, 50% of councils (200?) on probably 6 massive enterprise solutions and repeat every 4 years.  The sums will make your eyes water.

Of course this is not the case in every council, or every type of council but this tendency to Balkanize for survival is real and is costly. This is the reality of how the status quo is maintained. IT managers, CEOs and elected members are aware of this.

Seemingly Mr Pickles is not, or he does not care to admit it. Last week at the LGA Conference he reportedly said:

Back office functions should be shared: "Is it really right, in this day and age, to have separate planning departments?  Lawyers?  Communications teams? Wouldn’t it be better if people were working together? That’s especially important for the highest levels and the most expensive people."

Risk aversion

The second trait exhibited by local councils when purchasing IT is that of risk aversion.

This is very true of small ticket items.  No matter how small the spend you propose, the immediate question is "who else is doing it?", and if you cannot find any other known (trusted? lucky? talisman?) council sporting this product already - then you know you have a strong chance of losing the (hidden) argument between squabbling members/officers and losing your case.

Ironically, being able to say "Ah, but Council-A has already tested and started using this software!" can be both your trump card and the kiss of death as you play to Risk aversion but incur the wrath of Fear of merger.

Safety in numbers

The third truth about local government IT involves the very powerful organisation which represents them as a whole.

SOCITM is a private company which represents IT managers, it might help to think of it as a kind of union. I cannot say I know much about SOCITM because they operate like some kind of black box - unaccountable and opaque - no-one knows what is going on behind their closed doors.

In the last 12 years they have increased their fan-base and revenue by setting themselves up as judge and jury of what is both acceptable and recommended on local government websites - enabling them to recruit a legion of web-team and marketing types. 

I have never been a fan of the value of their content or advice as I have documented elsewhere. They just don't have the IT chops, and in fact are partly responsible for many of the lumbering and dumb local government websites horrorshows we see all around us. They are quite capable of emanating conflicting, nonsensical information and neither feel the need to admit nor apologise when they get things wrong. They just deftly repackage and repeat all the things that were talked about on the internet last year without adding anything new, and silently ignore the new elephants in the room which everyone is asking themselves about now.

When they make a mistake, when they miss important movements and swell changes on the web their effect on local government, the web and the country is crushing.

SOCITM seems to consists of sharp suited consultants and a sharp tongued marketing machine well versed in extracting public money from councils, they know exactly how to play the fear card.

The sucker punch

This week saw a SOCITM masterstroke pulled right under the noses of the new government. They have privatised the what was until then the open but little known e-Government Register (eGR) and stuck it away behind their own members only paywall in order that they can "cherish and look after it" on behalf of the British people.  "Closed Data Now" SOCITM does a "Times".

Now let me remind you what was the eGR (e-Government Register) was, put it in context and explain why it is relevant to these three previous local government IT truths. You could skip the next few paragraphs if you read the previous story.

The eGR showed which Supplier was providing which Product to which Council.  It provided for example transparent evidence of the Fear of merger and Risk aversion patterns - so you would have been able to argue with me using facts, but no longer - because it has just gone.

Over a period of around 6 years the screens on the eGR had been improved and added to until one was able to pick a Product and see which Councils used it, pick a Council and see which Suppliers provided them with Products. It had been further enhanced with nice but unnecessary Google maps so that you could appreciate the geographical spread of products throughout the country.

But, as an SME I can attest that it also provided a good overview to the cracks in supply. I could see where all the big suppliers were concentrating their efforts and where there were opportunities.  One of the small products I maintain is a Committee System a SAAS which manages Committees, Members, accessible agenda and minute documents etc.  I could see who my competitors were, to which councils they were supplying.  OK so the information may have been out of date, but I could see that less than 20% of councils were using any system at all.  This eGR should be a great driver for small software houses to develop those important niche products which could offer savings to councils.

Surely this is the kind of thing that the UK government should be fighting on our behalf to keep open and free - not subjecting me to a SOCITM tax of £795 per annum.  Most SMEs will not be aware of it, and I doubt any will be willing or able to pay the exorbitant fee.

The old eGR was getting out of date - hardly surprising seeing as few knew of its existence.  I would take a guess that nobody in the present government knew of it - how else can you explain the scandalous disappearance of this existing open public dataset at a time when they are lauding the opening of new sets.

To say that "the public would not be interested in this information" is to totally miss the point of the Open Government Data movement, the public have a right to view this information, and that right has just been taken away and handed to a closed-shop union which only stands to benefit from maintaining the status quo of a Balkanized local government IT scene.

It is the equivalent of Maggie Thatcher coming to power and handing the responsibility for reporting on coal stocks to Arthur Scargill.

To say that we do not have the wit to maintain a database the schema of which the majority of the web programmers I know could draw up in their lunchtimes, locate and match it to a taxonomy for local government applications and host it in the cloud somewhere means we really are up a creek without a paddle.

Gone is the chance for hundreds of small specialised software houses to apply their agile skills to solving local government problems because they will not even be aware of the opportunities.

Gone is the ideal resource that could have been used to de-reference suppliers and individual services mentioned in upcoming Linked Financial Open Data triples has been pulled out of circulation.

Most importantly and most ominously, GONE is the one thing that could have answered the question, "Blimey, Mother, have you seen this: Payment £245,000 by Council-C to Supplier-Z - FFS! what the HELL was that for?".

Moves are afoot to force councils to publish all spends over £500, but it requires another data set to answer the question what does that Supplier supply to that Council? This so called "de-referencing" by linking is something which I can tell a machine to do for me, or you or the bloke asking that question.

With the eGR in place the army of h-activists be they working for transparency, democracy, accountablity or looking to explain to people what money was being spent on, journalists, do-gooders or trouble makers would be able to link payments to applications and start to make interesting comparisons our our behalf.

Given my evidence above you can see how some of those comparisons could lead to nervous glances in some councils.

I remind you, the eGR was there last week, this week it has been locked behind the SOCITM paywall, with the knowledge and help of people in local government who, er, fail to fully grasp the severity of the situation or felt sufficiently hogtied so as not to put up a strong enough defence for this already public data set.

So what is SOCITM playing at here? Are they a load of dummkopfs for not realising the value of the eGR in outputting LOD or are they cunning masterminds for withdrawing from public view the one piece of valuable evidence which identifies which council pays which supplier for what purpose?  Is this the amazing service they are providing for their "members", tightening up the drawstings on the sack to make sure the glare of public scrutiny can never illuminate inside for fear someone might demand some searching questions?

I suspect it is the former - the evidently capable and ruthless but IT challenged SOCITM marketing department out to make a profit but it could be the slamming sound of ranks being firmly closed in order to protect their real paymasters  - big IT companies.  Which is it?

Thanks to a mix of greed and/or stupidity a huge pillar of the evidence about what goes on in local government has just been sidled out of the door.

Are you angry? If you are not angry then somehow I have not explained myself properly, so please go ahead and ask a question. If you are angry then please mention this post to your MP, blog or tweet about it. Please.

Maybe this issue has not yet appeared on the young government's radar and they will now take some positive action to rectify this deplorable situation. I don't care about acknowledgments, justifications or arguments. Just let the correct data emerge on data.gov.uk well before the Jan 2011 deadline for councils to publish their spending data.

M/billions spent on local government IT systems some not fit for purpose, talk of cuts to front-line services, massive public debt, the movement for open data and a government minister who can talk the talk vs ingrained fear of merger, risk aversion and a powerful quasi union whose marketing department has worryingly convinced many in government that it is part of the solution.

Gonna be an interesting one to watch.

Now, can we have our data back please? This time I'd like it sorted against a recognised and acceptable taxonomy.

If #localgovweb supplier says "RDF WTF?" Sack 'em #opendata #spending

The challenge: Linked Open Financial Data, put your council spend over £500 on the web by the end of the year.

It is really early days, but people are discussing what format that data should ideally be in when published. There is encouraging talk about which taxonomy should be used to describe payment types and of course the level of granularity which should ideally be produced.

In line with TBLs 5 star checklist, there are calls to JFDI, just get it out there and get yourself 1 Star.

Some brave councils have put some figures out some for honest and good reasons, others 'because they were told they got something for nothing', sigh.

There are h-activists doing amazing things with the bits of data dribbling out. It is all good stuff, and fascinating to watch unfold, and even I have chipped in here and there.

It is all gathering apace, and its all as you might expect - a bit of a free for all. I'm not knocking any of these tremendous efforts at all.

But I am taking the time to post a message - hey, Time Out!  Take a look at this idea.

Heres the situation:

It is 10 days now since Chris Taggart and Vicky Sargent published a report on lessons learned about publishing local open data for the Open Election Data Project.

I contract worked in Local Government for 10 years but still, I was pretty shocked by the no punches pulled tone and content of this report. I was pretty sickened and appalled to find out just how widespread this rot is.  I had sort of imagined I was just unlucky finding myself working in this type of environment.

Their report paints for me the all too familiar and glum picture of technological pygmies and naive decision makers falling for "throw money at it till it goes away" solutions. The tsunami of e-gov monies doled out over the years helped diminish in-house skills a long time ago, leaving many IT rooms in local government deviod of skills except how to put in support calls to system suppliers, or fiddle with a template in a CMS.

Honestly, you really need to read that report.  Go on, I'll wait.

Its not all councils - OK? Before you all look around for ropes and lamp-posts, there are individuals (and maybe some teams), in some councils who are have the where-with-all, political influence and skill capable of delivering top-notch results.  If you have any of these, then cherish them.

So, overall the situation is bad, right?

For me there is only one real answer (apologies if you heard me say this before)

Mandate the suppliers of financial software products to produce a 5 Star LOD (Linked Open Data) addition to their products by a reasonable time.

Starting NOW don't sign up for a Financial product without stipulating that you expect an open LOD stream with say 3 Stars minimum to be created from that backend system within a reasonable time frame.  Write it in the contract.

In tenders for Financial Products award 25% of the points for the ability of the supplier to deliver LOD.

If they try to charge you extra for that addition to their product threaten to cancel the contract.

I don't want to turn this into a tech post, but for the most part all you are asking for is a dump from their (your!) databases, formatted as a special kind of xml (OK, I'm dumbing this down slightly - the blueprint for the ideal format does not yet exist, as I described above).  If your supplier looks at you and says "RDF WTF?", sack them.

The need for Linked Open Financial Data is a technical problem for your IT suppliers - it is not a call for your HTML template author to go and buy a book on RDF.

5 Star Financial LOD has nothing to do with your council website CMS, unless you decide to stump for that nutty bastard kid of RDF, RDFa.

So - OK, suspend reality for a few minutes ... go along with me - now who are they, these suppliers?

Well, ahem, its pretty easy today, you go to something called the e-Gov Register (eGR) and you look them up.

Now this is where my knowledge of Financial IT systems lets me down, so forgive me and just get yourself on over to the eGR as long as it is still there of course and do a better job than me.

OK, it seems to me that of 450 or so local government organisations, 357 are listed as having a "Financials" supplier **. 

There are only 18 suppliers listed, and of those there are 6 Big Ones.

Between them the 6 Big Ones supply "Financials" to 326 Councils.

2

Don't you think that the first one of those 6 Big Ones who natively supports LOD as an export option  (or agrees to within, say, 8 months) really ought to be favoured when bidding for new business?

Can you say "competitive edge"?  How long do you think it would take the other 5 to catch up and reword their contracts and get programmers on the case.

You should think in terms of minutes rather than months.

Lets go further, lets say that it should be mandated that all new contracts with "Financials" suppliers include an LOD clause.

Perhaps Mr Pickles could dispatch someone to have a chat with one or two of these suppliers, or that he should have someone check that future contracts for Financial products being sold to Local Government all contain the necessary wording to make this happen?

So instead of trying to train and cajole 450 councils to FTP assorted CSV files into localdata.gov.uk (FFS) all the way through to grokking RDF, namespaces, LOD et al - why does the government not get on and make a strategy to bully and coerce 6 suppliers instead - and potentially get 326 councils teed up to produce useful LOD a bit sharpish?

Christ knows,  we pay them enough public money (reaches for his armchair auditor, now how much would that be then?).

That is my radical idea. Please drill holes in it if you feel the inclination.

If you find the eGR to be a fine thing and a good example of useful, publicly available open data which has been sitting there for about 6 years or more, and are wondering why it is not there now - then it has probably quietly slipped behind the SOCITM paywall and you should be angry with them about that, ** even though the top 6 suppliers may now be out of date.

Let them know in the comments on my blog, because if you try to add it to their "corrected" version of events on their blog it might get "mediated".  The appearance of some soc puppets would not surprise me in the least.

"Closed Data Now" SOCITM does a "Times"

As we rush towards the "Raw Data Now" door, but it seems few are aware that a tranche of historically open data is quietly slipping behind a paywall.

If my memory serves me well the e-Gov Register (eGR) hosted by Brent has been showing every IT supplier sortable by product type, supplier, local government type and even on maps for about 6 or 7 years (some links below if you hurry up).

I am aware that there are problems with this data, in my own past employer I know that some of the data is out of date.

But it is there, it is useful and informative and it is OPEN to all, even SMEs like me researching on niche markets in local government.

The latest move by SOCITM (and presumably with the knowledge of the LGA and the IDeA) means all that data is going to go behind the SOCITM paywall.

Just about every council in the UK has little option but to pay SOCITM hundreds of pounds annually to join their club to find out the exact details of how their website is being ranked. 

That is public money, our taxes - and has always enraged me.

And now they just found another way of ramping up their bills, and creating themselves another revenue stream - by hijacking what is already openly available data and reselling it back to us.  Here is the explanation hosted on the Brent site;

"The Socitm Applications Register will be created from a merger of data from the eGovernment Register and the Socitm Application Software Index.

The eGov Register will cease to provide Supplier, Product and Usage information as from 28 June 2010.

The new Socitm Application Register service will be available from July 2010.

The key difference between the two systems is that the new service will be available on a subscription basis. It will be free for public sector users who contribute data but other users will need to subscribe in order to gain access. "

I wonder does this mean that you have to be a paying SOCITM local government subscriber to be able to access this information, much like they only let you access "Better Connected" details if you are a fully paid up member?

Anyhow, bad news for any other small businesses out there, likely bad news if your council is not a paid up member and bad news if you are a council employee not lucky enough to have been given the privilege of a SOCITM account - or gasp, are not even aware of SOCITM, nor the e-Gov Register and were wondering what systems your neighbouring council were running.

Wizard worker on your councils GIS system? Fancied a move northerly, wondered which councils ran the same systems?  Out of luck now I am afraid!

Take a look at some sample eGR pages, they won't be there for long.

eGR Product types

eGR Financial Products, market share by supplier

"Free our data now" am I the only one to find this laughable hypocrisy taken from an article in elReg just too much to bear?

Jos Creese, the newly elected president of Socitm and head of IT at Hampshire County Council, said: "The momentum behind the 'open data' movement is gathering, and councils that get involved now, with this elections project, will put themselves at the cutting edge of this movement."

Won't it be great when all the Linked Open Finance Data is finally out there and we "armchair auditors" can see exactly how much each council pays to SOCITM?

(download)

Update 6th September 2010 The eGov Register is now the subject of an OPSI unlocking request - please add your name and support the handing of this data back to the public where it belongs.

Tagged egr socitm

TweetMiner saga: Good Guys 2 Lions 0

I really like Justin Vincent.

I've never met him, or even spoken to him, but I really like him.

I first came across Justin when I came across an OOP DBAL (Database Access Layer) he wrote called ezsql (strapline: PHP and working with Databases for the lazy sod).

This library was a godsend to me at that time ( PHP4, no PDO a myriad of other DBALs were floating around).

I used ezsql on projects all over the place and it did several important things:

  • it was well written and commented and provided minimal but accurate documentation
  • it abstracted away my database connection and database queries (of course)
  • it made me start to think in terms of classes and objects
  • it simplified my life

Ezsql played a really pivotal role in my learning at that time, and only now have I finally felt the courage to extract all the hidden dependencies on ezsql in a monster CMS project I am currently awakening and murdering for the umpteenth time.

So, Justin made some software available for free, and it was important to me and many others besides - for me Justin was a good guy because he embodied the spirit of OSS generally, but PHP in particular.

A few months ago I came across Justins name again, now he is the US ( #todo factcheck this: I think he was in Ireland before ) and he has a new product called TweetMiner.

TweetMiner looks very clever and is a "freemium" modelled web app built upon Twitter, and I can see how that will play into my own marketing strategy later this year.  Its bookmarked and I keep a watching brief on it, plus I follow @justinvincent Tweets in any case.

Justin's TweetMiner has ~3000 customers and around 100 paying customers.

So I was pretty horified to hear when Twitter suspended his account - leaving all these users in Limbo.

Read about what happened, blow by blow.  Read about how it was resolved.

Short version: technically Justin broke some of Twitters rules, one in particular, which when abused could lead to TweetMiner being the source of Tweet spam likely following this post by "... a leading web strategist for corporate investor relations".

Now here is the amazing part.  Justin asked the Twitter team if they could bend their rules in his case.

The Twitter team wrote back in effect saying, "OK, go on then, we will in your case - but its going to be up to you to police your clients."

All of this warms the cockles of my heart, Justin, clearly a good guy in my book, pleads his case and the Twitter team sensibly and with humanity acknowledge him, his users, the service he provides and suggest back to him a solution which devolves responsibility back to him.

I goes without saying that naturally I expect Justin will in turn delegate responsibility back to his own clients, and he will also find some clever algorithm to make sure that he those rules are policed and applied so that he sleeps well at night.

Now it could be that Justin's case has been heard by the Twitter team before and he simply went into a marketing  funnel and out popped this reaction, but I don't think that is true.

For me this display of flexibility and respectful behaviour is a deeply positive sign of good quality management in action on Twitter's part - and augurs well for its' future.

Bravo Justin and Bravo Twitter.

Follow Justin here @justinvincent here tweets daily using his own system, and you will find the signal to noise ratio is very high if you have even a single entreprenurial bone in your body.  A great mix of tech and startup skills.

http://tweetminer.net/

Update: Justin rebranded tweetminer as "Pluggio" www.pluggio.com

Tagged PHP

/*TODO read about simple comment toggler */ #php

Yesterday I read another of Chris Heilmann's great posts ( @codepo8 - a man worth following ) entitled the 7 Deadly Sins of Javascript Implementations.

Its a great read, but hidden in there was this gem of trick on how to very simply hide/turn on comments around a block of code which you can use in your PHP development.

As Chris says: "This trick makes it awfully easy to toggle whole blocks." and it is worth re-iterating that this technique is most useful when debugging or developing a script, and not a means of leaving loads of old comments lying around in your code.

Instead of writing comment blocks like this /*  stuff   */ you combine it with the order of the other commenting style "//".

Heres an example  Remove the first slash on the first line and the block is suddenly removed from programme flow.

//*

function do_footer( $identifier, $comment_type ){

    print ("</div><!--end main-->\n");
        include ("news_bar.php");
        include ("bot_nav.php");
    return;
}

// */

Then in the comments of the above blog-post, Ricardo suggested this improvement :


/**/
“live” code
/**/

/** /
commented code
/**/

looks nicer and is more obvious :)

I found this Ingenious, but then again maybe I am easy to please...

This was thrown into higher relief for me as last night my current required reading is the chapter on comments from Uncle Bobs' "Clean Code".

My god, do I feel liberated from the tyranny of docblocks after reading that chapter!  (big thanks to @AnthonySterling for the kind donation of this book).

(EDIT: the same day)  I just realized where I would use this technique a lot, in my UnitTester - the front-edge of my UnitTester (ie the bottom) often has me noting or stubbing out the name of the next few tests, especially if I am thinking of taking a break.

/**/ testEmptyArgThrowsException(){} /**/

I would typically have 3 or 4 "ideas" stubbed out like that, a kind of forward notation for myself - I inevetiably either don't use these stubs or change their name/intent as I work my way out.

Tagged PHP

Static Gmaps even easier to achieve : Usability 1 Lions 0

Hey you local gov webbies, using Gmaps to get jiggy with your newly leeched data gov stuff?

Well, there was an announcement last week you might have missed on the Google Geo Dev Blog, Static Markers just the way you like 'em.

In case you did not know what static maps are (shame on you!), they are like a snapshot of a map area you describe and can contain markers - so they look like your dynamic map - but contain no interactivity - its just a picture.

However at least people behind some firewalls, those without JS turned on for whatever reason, can at least see a representation of your map.

All you have to do is assemble a carefully crafted URL inside a <noscript> tag. Static Maps V2 API Developers Guide.

EDIT dont forget, put that <noscript> tag inside your map <div>

<div id="MapDiv"></div> what you likely have, a place holder for Gmaps to do its thing

<div id="MapDiv">
<noscript>
<img src = " ... " />
</noscript>
</div>

The above announcement quite rightly bangs on about how you can now produce static maps using your own markers, it masks another bit of good news, you don't need to include your Google Map API Key any longer - so one long string less in your markup - or one dependency less if you were getting your Map API Key from an include file - one less thing to go wrong, and one more reason WHY YOU SHOULD use a static map fallback option whenever you create Gmaps.

(unless you are carrying round the API key in every page on your site regardless of whether it carries maps or not, surely not?)

Maybe its not huge news, but its a nice move and one that I appreciate seeing as I have about 6 different templates calling Gmaps that I can refactor and lighten up and match the Static map markers to the Dynamic map markers.

Get it onto your jobs list and JFDI, eh?

Short version: You can throw up a map on any page as long as you can get some coordinates, or an address that Google can decipher.

Bad News Alert: Wine makes you fat

'Fraid so people.

Someone had to tell you.  Here's my proof and some before and after photos (not of me, you'll be glad to hear).

Skip the boring bit if you want.

<boring bit>

My wife and I follow a fairly non-invasive eating routine (I don't call it a diet) where you can eat what you want but you pay attention to what you eat with what.

Fruit AND vegetables is out so are mixtures such as steak AND chips, cheese you only eat at lunch time, etc.

Anyhow, this routine has worked just fine for a few years, and immediately stopped my awful midnight bouts of stomach acid I had for years and years.

Still, I was gaining quite a paunch, so much that I no longer relished visiting a beach this year.  I am pretty slightly built, not at all muscular and only average height, so 75 kilos equated to a midriff of 100 centimetres.

So I decided to do something about it.

On July 14th this year I started a diet.

I simply stuck to our normal routine, but stopped eating stupid shit.  You know what I mean: biscuits, cakes, desserts, snacks, cheese sandwiches between meals... chips, crisps, aperitifs etc.

Breakfast on our routine has been only wholemeal bread and honey for a few years, pretty hard to wake up to face that, but I have reluctantly gotten used to it.

At lunch I ate a salad only, or something very light, and a slice of bread.

That wasn't hard to do - and by the end of August (6 weeks) I lost an admirable 5 kilos - from 75kilos to 70.

I was pretty chuffed, and had to start hunting for belts to hold my trousers up.

I thought to myself "This is going so well, that I'll be at my (secret) target weight of 66 kilos by my birthday in mid-November". 

Ah, not as easy at that as it happens.

For the next six weeks I stuck rigidly to my diet, and to my surprise and dismay I lost no more weight.

With one month to go before my birthday I decided I'd better get serious with this fat.

</boring bit>

So on the evening of the 18th of October I announced to a shocked and tickled world that I was not going to touch a drop of alcohol till my birthday, a month away.

I stuck to the same diet, but no alcohol passed my lips.

On my birthday I weighed in at 67.5 kilos, so in a month I lost a further 2.5 kilos.

Ergo, wine makes you fat.

Now that is bad news when you live in a major wine-producing area and you have a taste for wine.  Plus, I know where I can buy a litre of very, very nice local wine for 1 Euro and bring it back and bottle it myself.

I might be slightly cheating here, but I did take one other action for that month, I trimmed some pine trees for an hour most mornings - instead of walking the dogs for an hour they variously lounged around watching me trim pine trees, or took of and had fun doing whatever it is dogs get up to when left to their own devices.

If you like your wine that much, then you can use the above paragraph as a get-out clause, and continue to imbibe in denial.

If you want to know what that paragraph looks like in practice then here's some before and after photos of the lines of pine trees I tackled.  Only a couple of thousand to go.

Why do this?  Well each year the pine tree pushes upwards and adds a "crown" of branches at the top.  By trimming the bottom crowns of branches ( usually 3 or 4 rows every 3 or 4 years) the energy in the tree is diverted upwards, so prompting the growth of a stronger, straighter tree.

As the bark will heal itself over time where the old branches were cut out, so when the pine is harvested and sliced it will contain less knots. The tree is also easier to clean around and perhaps most importantly, we (including the dogs) can now see through the trees to the other fields beyond, so there will be less wildlife, less deer attacking the bark of trees and less illegal hunters hunting those deer - because they will feel far more exposed.

So, stronger straighter trees are more valuable, and illegal hunters are pissed off, and I lost a bit of weight. Result eh?

Postscript: I put the lot (2,5kilos) back on between my birthday and New Year).

(below: 4 photos before, 4 photos after, 2 photos of the sticks I cut out from the branches and saved for the fire this winter)

(this post inspired by @AnthonySterling an old Sitepoint mucker who seems to follow my twitter gushings)


(download)

Google OCR - sorry, no cigar

Tony Hirst tweeted about Google OCR about an hour ago :

@psychemedia  
 
Ooh - didn't know (or completly forgot about) that one - a google OCR API http://bit.ly/7QQP8c


So I thought I'd give it a swift test ... a snap of part of my bookshelf actually.

Results below, not too good really.

Never mind, nice idea, send it photos and it might have told me stuff written on them.

That was pretty much motivated by fun I had yesterday looking at old photos of Godalming uploaded to the newly created "Godalming and Farncombe" Flickr Group Pool.

This one of Church Street Godalming is great ... a snap is below, along with the book scan and its Google OCR result.

Take a look at the big sized original on Flickr though, check out the shop sign on the LH side.

E.WEALE
Cycle, Motorcycle,
Wireless & Pram works

LOL, wireless and pram works in 1937! Truly multi-tasking.

http://www.flickr.com/groups/godalming-and-farncombe/
http://www.godalming-tc.gov.uk

ps if you have any photos of Godalming and you're a member of flickr, please let me know if you want to join our group or donate any photos, new or old, any subject we dont mind.

(download)

Tagged google ocr