Any comments/feedback on DataShine? Post your thoughts here, and we’ll do our best to reply. The most recent comments are at the top, and a box to leave your own is at the bottom of this page (jump to the box).
You can also email: o.obrien [at] ucl.ac.uk
Hi great product. Will you be adding 2021 census data, if so when. Thank you.
Tony
Hi. Thanks for this resource. It’s brilliant. Are there plans to publish a new version for 2021 census data? If so, is there any indication when it will be ready please? Many thanks.
Hi,
I am a transport professional (graduate of UCL too actually!) and would like to use this data to look at the distribution for development traffic. I can plot the data and the table appears in the bottom corner but how can I establish the points of origin please? I have selected my site as the end destination as we are looking at employment space.
Thanks
Lizzie
The MSOA kmls don’t work anymore. Wondering if it wouldn’t just be easier to have the MSOA boundaries as a layer?
I agree, it’s a well designed tool! I just have a question concerning the residency tab, are you sure the figures are correct? (or maybe I am missing something). Thanks !
Age 15-24 is missing from the OAC variables
Wow this a great tool! Do you know when there will be one for Census 2021?
I hope so – although bear in mind the Census statistics, in the detail we need, won’t come out for typically a couple of years after the Census itself.
Hiya,
Love this tool, will there be a similar project or updated datashine from the results of the 2021 Census in a couple of weeks?
many thanks,
Nath
I hope so – although bear in mind the Census statistics, in the detail we need, won’t come out for typically a couple of years after the Census itself. The ONS also is I think planning to do a similar thing.
Having tested on two devices, the display for the England and Wales map seems to be broken, at least for me, not displaying roads or buildings, making it difficult to use the map. I think for some reason the “Houses” toggle is broken. The Scotland version still works for me, however.
Hi, there was a certificate issue for a few days around New Year which would have caused this, apologies for that.
I am not seeing any roads on my iPad macbook air or macbook pro – all different ages newest 2020. Can you help please?
Hi, there was a certificate issue for a few days around New Year which would have caused this, apologies for that.
This is a great tool *BUT* it has a tendency to produce some VERY distorted scales.
Eg looking at BAME populations for Brixton area of Lambeth – specifically Black/African the ‘default’ sets the bright red band at 11.5%-100% so nearly all of lambeth appears bright red. Space equally obviously needs to be there but that doesn’t really show highlight the differences.
Obviously ‘resale for current view’ is meant to automagically do this but it would be much more effective if the scale could be manually set (or with a slider) to get to a view that best highlighted what the user was trying to show – what’s needed is something between the automatic one and the ‘default’)
I totally agree! Getting a one-size-fits-all system for showing the bands was tricky and was always going to result in issues like this. These days, the banding can be done a bit more intelligently.
Thank you very much for the DataShine Commute website which is extremely useful in helping our local authority clients to understand commuting patterns. Could you please confirm whether it is possible to reproduce screenshots of the data visualisations in reports that we produce for these clients, with a suitable attribution? We are a planning and environmental design consultancy.
No problem. To cite the project or websites, please use: Oliver O’Brien & James Cheshire (2016) Interactive mapping for large, open demographic data sets using familiar geographical features, Journal of Maps, 12:4, 676-683, DOI: 10.1080/17445647.2015.1060183
Hi I’m loving DataShine as a geography teacher – but one issue – OAC variables doesn’t have 15-24 year olds – why not and how can I get full range of population age categories ?
Cheers
JP
I’m guessing that the variation of 15-24s in the UK wasn’t sufficiently varied in order to be a useful variable in OAC. Unfortunately this is not available elsewhere in DataShine.
Is there any way to get the map for population density?
Thanks
Yes – but on CDRC Maps not DataShine. See https://maps.cdrc.ac.uk/#/metrics/ruralurban/
I’m sorry to be dense. I’m not a statistician, i’m a local Councillor trying to understand how many people travel into our town ( Lewes, East Sussex) for work. I added up all the totals in the box from the blue lines and got over 6,000. That seems too many for per day, and not enough per year. Please help.
By the way, thanks for doing this, I’m sure it will be really helpful when I get to grips with it.
Hi, it is daily. Also, note that “showing maximum of 1000 flows on the map. Only flows with 6+ people are included.” The smaller flows can be significant in some areas of the country – they can represent up to 40% of people. Bear in mind also that Lewes has 2 MSOAs – so don’t count the flow from one to the other (or vice versa).
Also don’t include same-MSOA flows either!
LOVE THIS. Great tool. Can’t get it to work on my iPad Pro (‘Go’ box doesn’t work), But aside from that a really, really helpful way of analysing data. Thanks.
Hi Adrian, yes the website predates the rise and rise of iPads and other tablet devices I’m afraid, so wasn’t built with them in mind.
I added some overlays using google earth’s kml files (great job on supporting that btw!) however when I save the pdf, it saves without the overlays.
Any chance we could get a feature that takes overlays into account when saving the pdf?
Many thanks
Hi Florian, thanks for the suggestion, I’m afraid DataShine is archived now but hopefully it will evolve again one day.
I can’t find Russia for the country of origin category
Hi Lois, Russia is included in “Europe: Rest of Europe: Other Europe” along with a number of other countries where the population is small (or was in 2011 at least) and therefore are not published seperately at output area level. See http://datashine.org.uk/#table=QS203EW&col=QS203EW0031
There is a Country of Birth (Expanded) dataset which has Russia seperately, but this is not published at output area and so was not included in DataShine.
This is an awesome resource, many of our clients have been busy exploring the census data. The presentation is awesome.
It would probably help you from a confidence and SEO point of view were you to implement an SSL/TLS certificate on your domains (allowing browsing using https rather than http). In cases such as this blog where personal data is processed, a secure certificate is now required to be compliant under GDPR.
Secure certificates can be obtained easily and for free from LetsEncrypt (https://letsencrypt.org/)
Hope this helps.
Thanks so much for making this resource available.
Very impressive visualisation of the 2011 census data.
Suggestions for small improvements.
It would be nice if you also linked the 2001 census data to these maps so people could compare the change. Ideally allowing users to click between 2001 and 2011 to easily see where places have changed. And where data sets allow the 1991 data. I am aware that in 1991 ethnicity data was collected but religious data was not.
The Consumer Data Research Centre, have visualised some of the census data as well. They allow users to scroll through a time series of maps, showing estimates from 1997 to 2016.
It would be nice to see the best features from DataShine and the Consumer Data Research Centre combined together.
A wider choice of colours would be an improvement. Allowing for more colour combinations for the two way heat maps, rather than just Green to Red.
A three way heat map, using Blue, Red and Green as the primary colours would be useful. This would help when comparing two main variables and a third given as others.
An example would be.
Blue = Christian, Red = Secular and Green = All other religions. The individual colour given to each area would then depend on the % of Blue, % Red and % Green.
Reading the commuting data, it may state for example that 32 people arrive by bus from a location.
I am unsure how to interpret this. Is that 32 people per day?
Many comments are made regarding people owning second homes in Cornwall. It is my experience that the majority of these “second Homes” are in fact homes owned by people living in the area, or now abroad, and using them as revenue generators – ie holiday lettings. Your census information does not enable that enquiry. It would be useful to identify second homes that are in fact a business, or those that are owned by local people who do not in fact declare them as business properties, but use them as holiday rentals.
Hi,
This seems quite useful but I just have one question related to the accuracy of the data. I was looking at ‘Origins & Beliefs’ and I wanted to see the majority of different beliefs in different places for example, I selected the ethnic group as ‘Black, Caribean, Black British’ and map showed high density upside of Welling station so was just little curious if this data is uptodate and correct because when I looked at the same ethnic group on ‘maps.cdrc.ac.uk/’, it showed no group has majority at that place.
Can you please confirm the accuracy and if this is updated or not.
Thanks in advance.
Vik
Firstly, this is a very useful way to present the Census data in a more intuitive and comprehensive way.
Looking at the link for the TTW element to glean details of origin and destination the only thing that i think would be helpful would be if it was possible to display the ward boundaries as well as the other information
What does ‘economic activity’ and ‘economically active’ in the data chooser exactly mean?
Hi, I’m trying to use the raw excel data from your website to compare religion and education levels spatially in Edinburgh for a research project I am undertaking at the University of Edinburgh.
I am able to download the data for education levels in Edinburgh, but not for religion. When I try to download the excel data for either Church of Scotland or ‘no religion’, it comes up with a message saying ‘bad table parameter’. I’m fairly sure that when I checked a couple weeks ago, I was able to download the data without issue. Is there a way I can rectify this?
Thanks,
Hugo
Hello,
Thanks for this useful tool.
When I export PDF from the heat maps, the pdf shows percentages on the right side. What does ‘LQ’ on the key mean?
Are we able to use these pdfs in our reports, as transport planners?
Thanks.
Hi Rose, LQ is the Location Quotient and in this context is simply a multiple of the national average. So an LQ of 3 means people in this area are 3 times more likely to have the selected attribute, than the national average.
You are welcome to use the PDFs in your reports, we would be grateful if you cite us (Oliver O’Brien and James Cheshire) and include a link back to the DataShine website.
This is great tool that can make a lot of money for you! Don’t abandon the project. You can ask users for donations at this stage, try Kickstarter etc.
Great work and I am hoping you will build on this fundamentals.
Thanks, do see CDRC Maps which is a successor of sorts to DataShine, as it uses the same mapping style and maps the same kind of data (and more diverse types), although for a smaller range of data.
Unfortunately CDRC Maps don’t attempt to map religious data.
Less a comment than a request to publish a few samples of these maps in the annual of the Kensington Society, a local amenity and conservation organisation. I was hoping to show some screen grabs which would of course show your website address, with a brief note about what this is and what information is available.
Certainly, that would be fine. Thanks for asking!
Please note that the DataShine project is in legacy mode – we continue to keep the websites up but are not carrying out active development on them. In parallel, we endeavour answer questions about existing functionality in the sites but at this stage it is unlikely we would be able to add to or otherwise enhance them based on requests.
We are always interested in follow-on work and bespoke commissions, please contact Oliver O’Brien at the email address above.
The residence in the UK maps seem to make little sense – they suggest that few people in the Wirral area were resident in the UK at all (looking at all four categories). Or have I missed something?
Resident in the UK represents the population not born in the UK. Choose “Born in the UK” in drop-down 3. I appreciate this is a little confusing and that the two should really be combined together. This is a quirk of the way the source data was set out and we decided to keep strictly to that structure rather than do some obvious combinations like this.
looking at OAC variables for population age categories.
There isn’t a 15-24 category.
Would be very useful to add this
Otherwise brilliant website
This is an excellent resource! One option I would really like to see (in addition to request of Nick below) is selection of the output by more than one variable, e.g. education and employment. Is this something that you could make available? This would really enhance the practical utility of the resource.
DataShine Census only shows univariate data (i.e. varying in one dimension). See its successor CDRC Maps which does show some multivariate data, although only for a small number of datasets.
Brilliant!
This is awesome, thanks. I would love to be able to select more than one variable – specifying combinations like ‘percentage in the area that has education at degree level and less than 50 percent white’ Maybe with up to three or four sliders to select.
DataShine Census only shows univariate data (i.e. varying in one dimension). See its successor CDRC Maps which does show some multivariate data, although only for a small number of datasets.
AMAZING! Congratulations.
Can you also upload workplace zone data to this site too?
There is a Workplace Zone classification map in CDRC Maps which is a successor to DataShine: https://maps.cdrc.ac.uk/#/geodemographics/cowzew11/ – however Workplace Zones (& their associated census outputs) were only made available after the end of the DataShine project.
This is a really useful tool for us. Thank you.
Just one question – is it possible to drill down to postcode sector level?
Hi Jeff. Currently we only map wards and OAs. Postcode sectors are a different geography (controlled by the Royal Mail rather than the National Statistics agencies, and a commercial data set rather than open source) which is why NS data like this is not so easily published against postcode sector.
Dear madam/sir,
I wanted to know if it is possible to export your maps into Tiff or SHP (Shape files of Arc-GIS).
Please let me know if it is possible.
Mohammad – not currently (you could screenshot and save as a TIFF but that wouldn’t be a GeoTIFF of course). However it should be straightforward to recreate in ArcGIS from scratch, using the underlying source files.
Nice work! Would you consider making DataShine open source on github?
Hi Nick – certainly this is an aspiration and should the project restart (it is inactive now) then this would be a priority.
Great work folks – as ever the simplest change can make the biggest difference to users.
The ability to switch to a “house” based view – with thematic mapping representing the objects in which people live – away from the standard colour flooded Datazone view – enables users to better focus on their areas of interest.
Many won’t identify that the data remains the same and think a more granular dataset is being provided.
A further improvement would be to remove Business and industrial buildings (with no residential component). We have Datazones containing very few residential properties but extensive industrial areas. These extensive built areas are thematically coded with the result for the whole datazone (e.g. S01010619) – making the impact of the resulting output appear far more significant to the user than it really is.
A modifier based upon excluding buildings classified as non-residential (need access to Address Gazzeteer) or over a specific size (difficult with OSMM representations) – would help improve the output in this area.
Absolutely – would love to remove business/industrial buildings but building type attributes are not included with OS open data currently. A hybrid using OpenStreetMap landuse indications or other landuse information would be possible, however the former’s quality is not higher enough and there is not a high-enough version of the latter as open data currently.
I was working out population by age groups in England and suddenly this option has been deleted. Please can you let me know how to access age groups in a post code area?
Many thanks and I would be grateful for your help.
Hi Stewart, we have made no changes to the website which could cause data to be deleted. Age by Single Year is still listed. However if you are doing quantitative analysis we would generally recommend processing the source data files rather than interpreting our maps, that would be much faster/easier.
Are we able to use the output from this site for commercial work? It is a fantastic tool and I would be particularly interested in using the Datashine Commute tool if possible (appropriately referenced of course).
Please can you let me know?
Thanks
Fiona
Hi Fiona
It is OK to use screenshots from the website for commercial purposes, as long as the logos are not obscured, the URL to the site also included and it is credited appropriately.
Hi Ollie, Is there a toggle to turn ward boundaries on? When you toggle houses+streets off, it’s just a meaningless pattern of coloured shapes.
Hi Adrian, no, it’s a design decision not to include ward boundaries as their shapes are unfamiliar except to specialist/professional users and so would clutter the map.
Are we able to use output from the datashine site in commercial work if it is properly credited? Thank you!
Hi Eleanor
It is OK to use screenshots from the website for commercial purposes, as long as the logos are not obscured, the URL to the site also included and it is credited appropriately.
Is there any way to change the basis of the commuting data to aggregated town (locality) data?
USP” (Understanding Scottish Places) is based on locality data and although we already have some commuting data included origin destination data has not yet been incorporated. WE are thinking about whether to include this so I am asking you whether you envisage doing this or whether it is already available.
Hi Datashine team.
Thanks for developing such a great tool!
I have a query about the datashine census map – when I drop a KML file into the browser and export to PDF, the KML disappears. Is this to be expected? Is there any way of getting the KML to export to PDF along with the map?
Thanks,
Mark
Hi Mark – I’m afraid not, the KML remains on your local computer when you drag it to the web browser, so we have no way to capture it when remotely creating the PDF – however you can try Print Screen and this should work quite well.
This is a beautiful site! It generally works really well for us (undergrad GIS) – but when you download the data (CSV), what geography do you need to match it to? I have downloaded data by Ward from this site, but the ward codes do not match the 2011 Ward data available from https://census.edina.ac.uk/easy_download.html . Any pointers? The OA download matches the OA boundaries from the same site.
Also the spaces in the CSV column headers cause ArcMap to whinge, but this is easy to fix after download. Thanks for a great resource.
Hi Katy, could you give me a specific example and then I can investigate.
It seems I can find out the health, age, employment etc characteristics of individual households with this tool. ONS state census data is confidential. Are you contravening data protection/confidentiality with this tool?
Hi Philip – because we show individual houses, it can look like that, but you are actually seeing the averages/aggregated results across a small area – typically ~250 houses. We’ve assigned the corresponding colour across all the houses in that area. We use the most fine-grained data that is available as open data – this project does not map the more confidential/restricted datasets that area available at a finer scale.
Sorry if this sounds silly but I always feel like more colour schemes would be useful. Or at least, the ability to reverse existing two-colour schemes (e.g. a button that reverses red-green to green-red). Sometimes when looking at some census variables (e.g. negative health outcomes) it would be useful to map these as red and the complement as green. Many thanks!
Hello to All
Brilliant graphics illustrations can you please help me understand the quintonile thing and how it works in relation to color coding key and how far it is away from average have tried to work it out but very unsure
Ant help would be very much appreciated
Excuse my ignorance
It depends on the metric you are looking at. Some times the key is in standard-deviations away from the mean – “diverging”, sometimes it is multiples of the mean. The key should have the details. The data chooser also shows the current whole-country mean and standard deviation.
Thanks James & Oliver for this great data visualisation tool.
When are you planning to make the source code available? I have met several people across government, academia and non-profits who would be keen on using it, instead of trying to reinvent the wheel.
We would certainly like to make the code available that is not already available (mainly, the tile generation code) but there’s a big tidy-up/clean-up process that we would need to do so first.
Hi Oliver
Keen to lay hands on the source code of this super cool site. Please let me know when would this be available . Thanks
Vidhi
I’m sure that this is easy to use when you know how – but how can I get help with my first steps?! Help!
Just explore – the best way to learn. Most of the functionality isn’t hidden away, and we’ve kept the user interface deliberately simple so that, by using the website, you’ll learn about it.
this is amazing. my teacher showed this in geog class. please keep it up and keep updating. i presume you will do one for 2021 census as well so youve got a while untill you have to make the effort again!
Hi Sam, thanks! I certainly hope so – 2021 is a long way away but when the results come out I am sure there will plenty of data visuals appearing.
Excellent work.
One suggestion for Datashine Scotland Commute would be the addition of zone boundaries. This would be a big help when using DSC to look at rural travel patterns, data is very difficult to interpret without knowing zone coverage.
Hi Paul – you can drag a KML (or GeoJSON) of your desired zone boundary, onto DataShine – it will then show the area on the map. We like the KMLs that Mysociety Mapit produces – e.g. the KML link at http://mapit.mysociety.org/area/2601.html – download this and drag it onto DataShine Scotland Commute and it shows up nicely.
p.s. It’s interesting that Peebles/Innerleithen have a much higher number of commuters into Edinburgh than Galashiels/Tweedbank – look out for a blogpost about this shortly.
Hi Oliver, you’ve created a wonderful tool, thank you.
I don’t know if it was omitted intentionally but within OAC Variables there is no ‘% Aged 15 to 24’. Is the only way to view these ages using the ‘Age by single year’ in ‘Population Basics’?
Hi,
Awesome map and colors 🙂
I’m trying to make the same function “rescale from current view” for a project of webmapping but im a little bit stuck…
Do you have some clue to help me?
Best regards
Great resource, just wondering if it would be possible to zoom in more or highlight the appropriate polygon when you enter a postcode in the search box? At the moment when I type in a postcode it’s not obvious where on the map it is.
Dear Catherine
Thank you for your comment. Our postcode search facility is optimised for speed and efficiency, and is currently quite simple, it pans the map to the centre point of the first part of the postcode (e.g. for EH34 5AT, it will pan the map to the centre of the EH34 area rather than the location of the full postcode). Therefore, a marker showing this location might be some distance from EH34 5AT’s precise location. However we are considering enhancing the postcode search facility for a future update of DataShine, where the full postcode centroid is highlighted on the map, following a search, so watch this space. Note that for postcodes in very rural areas, the postcode centroid can still be a large distance from some houses that are contained within this postcode.
This is really excellent, congrats.
Paul
Hi there,
Amazing work, thank you
I wondered if it would be possible to do this for Scotland and Northern Ireland? The devolution of statistics makes this very difficult, but I wondered if you could comment on this
Many thanks
Gerry
Ah, I see your comment below now!
Would it be possible to insert another variable for the flow data? I’d be interested to have a cursory look at mode shift potential which would mean distinguishing between different length of journeys (as well as mode). Perhaps a five mile radius could be included?
Are you planning any capability to group census zones? For Journey-to-Work mapping – the true picture may be obscured by the >6 trips limit; that is, for example, for a rural town ‘destination’, if there are 4-5 trips from a number of zones in larger conurbations say 40km away (such that when grouped together may summate to say 30 trips from that ‘origin’ area) – then this travel pattern is not represented. The travel pattern would then look as if most journeys were from local areas (>6) – but journeys from the wider area may in fact be greater.
Gwyn – yes, indeed, the less-than-6 population becomes significnant for further-out areas. See Noel’s comment below and my reply to him. However if we apply automatic aggregation of further-away zones, then it becomes a more sophisticated tool and DataShine TTWF is designed for simple visualisation of major, local flows, rather than as a full spatial analysis tool.
Just a quick question. I am looking at the TTW census for Exeter and I am wondering if there is a key to where the different ward levels are located. i.e. is there a spreadsheet defining “Teignbridge 008” for example?
Hi Lloyd – I’m using MSOAs rather than Wards for the TTW, as that’s the most detailed geography that the ONS has provided for this. Teignbridge 008, for example is the ONS name for MSOA E02004208 – http://mapit.mysociety.org/area/38087.html
Hi Oliver,
I have found this very useful, working for my local council has given me a great insight into the movement patterns within MSOA areas. I have had a quick scan through the comments, I didn’t see any answers to my specific questions.
In an ideal world, the data would be there at the ward level, is there any plans in place to do this to ward level?
Also, there seems to be a large disparity between the totals I am coming up with, when I have used http://www.neighbourhood.statistics.gov.uk/HTMLDocs/dvc193/index.html#sty=true&flow=flow2&period=0&fix=E07000242&view=436.25,198.125,157.5,158.75&tr=0,0&sc=1 commuting patterns seem to suggest that 22,140 people commute into East Herts whereas this dataset suggests 15,000 commute to a MSOA from another East Herts MSOA and 15,000 commute in from outside EH, this is a 7,000 disparity between numbers. I see you have noted that numbers under 6 are not included but would there be any other differences that would explain such differences. There is also a similar disparity in commuting out of EH
Hi Noel, ONS have released the data at MSOA level, Ward level data is more problematic as some Wards can have quite small populations, potentially raising individual record disclosure issues. Looking at the data (and including the rows <6) I see a population of 22122 commuting into the 18 EH MSOAs. Without the <6 rows, I see 14698, so yes it looks like that's the difference. (the other very slight difference may be due to the ONS republishing the table with minor changes after we pulled it in.)
Awesome site.
The ability to look at age is rather limited. I’d love to be able to look at, say, what proportion of people of voting age (i.e. 16+) are under 30. Any chance of creating an interface to allow that? Ideal would be something where you could create any arbitrary age-banded ratio: define an age range for the numerator and another age range for the denominator, and then you could calculate any ratio you wanted…
– Dependency ratio (65+ / 18-65)
– ‘Youth adults index’ (18-30 / 18+)
– Kids density (0-16 / 17+)
… etc.
Hi Mark
I agree, the ability to aggregate some of the data together would be very useful and is something that we are looking into.
Great work on the site as a whole
I’m very keen on this feature too. Even the ability to do a simple logical AND between multiple selections on multiple datasets would be incredibly useful.
Is this something that any progress has been made on?
If so, when would you expect to be able to release it?
Really interesting and potentially of great social benefit.
I am interested in this from a public health perspective. I would like to map some of the core health variables – smoking, obesity, alcohol consumption, physical activity, down to the ward level or below. It this possible using existing data sets ? The PHE data goes to GP surgery level only and the census data you show is presumably self reported and a very broad, general impression of health status.
I am finding this information shown on the website very useful, however, it would be even more useful if you are able to hover the mouse over an selection output area and the name of the area was quickly shown without having to click on it.
Hi Raymond, I presume you mean on the Travel to Work Flows map? The background map should hopefully be clear enough, and have sufficient labelling, to indicate where you are. If you zoom in, more detailed labelling should appear.
I am finding the travel to work data from 2011 extremely interesting – especially in relation to thoughts about increasing walking and cycling Bristol. One problem I have bumped into is reconciling the 2011 LSOA and MSOA areas with data I have already looked at so (for one example) I can’t associate the centroids Bristol 054 055 056 and 057 with specific LSOA names that I already have some data for.
I wonder if there is a more up-to-date list of amendments than the one I might have been working from? Two of these numbers were listed among changes effected for the 2011 Census (054 and 056) I’m afraid I haven’t been very systematic in checking this. My main anxiety is that the areas concerned have very high concentrations of car use as workplace destinations and so are critical to the analysis I am thinking about.
Many thanks for putting this wonderful resource up for examination.
Sam Saunders
Hi Sam, if you hover over the name label, then the MSOA ID should pop-up after a couple of seconds.
The OAC maps are a revelation for looking at small areas where there is a concentration of social issues in rural areas, an absolutely brilliant resource for a rural community support organisation like ours. Is it possible to do the same pdf download from the OAC maps as it is from the census ones? This would make it much easier to communicate with Trustees, funding bodies etc the needs that the maps uncover.
Very many thanks for all this work
J
Hi Jeremy – A recent (January) upgrade to the DataShine OAC website means that you can now print the webpages directly, rather than needing to create a PDF. See http://blog.datashine.org.uk/2015/01/openlayers-3/ for further details.
I would love to re-use some of these visualisations in presentations but with three shaded areas taking up a quarter of the screen it’s very hard. I don’t mean to be ungrateful when you’ve done such excellent work, but is there a less cluttered option that people could use? Many thanks!
You can move and minimise the two blue panels – just click on the title bar at the top. Alternatively try the PDF map button, to download a PDF which you may find is less cluttered.
The data files on communal establishments are broken and return an error when opened in Excel. Please can you fix this?
Thanks!
Please could you paste in an example link.
Wondering if we can link or identify small areas within wards to the spreadsheet data?
Do you mean output areas? These are identified with the “area code” on the left hand side. If you mean LSOAs or postcode areas, stay tuned, we hope to have some enhancements which will show these, in the new year.
Oliver – congratulations on this fantastic tool. I’ve just been playing with the new re-scaling tool and making pdf maps (just for the hell of it). This is just the kind of thing that is sorely needed in local authorities. It is maintained centrally so can be used by all at no cost in software or maintenance and all users can take advantage without training.
My question – is it possible to flip the colour axis. I was just playing with the deprivation dataset and I noticed that areas with a higher percentage of households that are deprived show up as green, viewers will expect more deprived areas to show up as red.
Yes, the default colours are problematic – we’ve used Red-Yellow-Green as a default for when we want to show data that diverges significantly around both sides of the mean. We picked this one as it has a high visual impact, but yes there are issues with the colours having traditional meanings. You can use any of the other diverging colours available in ColorBrewer2 – just modify the parameter after &ramp= in the URL with your desired ramp. Unfortunately there is not currently a way to swap the ramp order around. For a more neutral one, try PuOr – or use one of the sequential ramps – buttons for some of these are in the panel at the bottom of DataShine.
Hi,
Just got the email about the latest changes…very helpful, particularly rescaling! If we could have a Local Authority and perhaps ward boundary this would meet many of our needs very quickly.
Stay tuned for further enhancements to this, which we hope to release early in the new year.
Hi Oliver,
This is a really useful resource. Do you know if data for Scotland will be available in the near future.
Many thanks,
Hi, I’ve run into some issues with Scotland – many of the Quick Statistics tables, that the England/Wales maps are based on, haven’t been released in Scotland – for example, primary languages or occupation/industry minor groups. Scotland does have Key Statistics tables though, so we may look to integrate these. But it does mean it’s slipped down the priority list for now. Northern Ireland is much less likely because of the non-availability of data similar to Ordnance Survey Open Data Vector Map District’s building outlines.
I would echo comments on travel to work data – in particular the ability to exclude not-at-work/unemployed to focus on actual trips made and a way to aggregate modes (i.e. cycle plus walking, rail plus metro, etc). Otherwise excellent!
Two things:
Firstly, can we use your mapped data in commercial reports we write, PDF’s or screen grabs?
Secondly, what data source did you use to match your E00 codes to the the name of that particular area?
Ta.
Hi Colin – yes, commercial use with appropriate attribution (making it clear that it’s not your own work) including a link to datashine.org.uk and that it was funded by ESRC, is fine.
To match E00 codes, aka Output Areas, to the area names (which are Ward names) we used one of the Lookup tables available on the ONS Geography Portal. Most OAs nest within a single ward even though strictly it’s a many-to-many mapping.
As per comments on twitter earlier, for the census TTW map it’d be useful to be able to group several modes of travel & an option to manually set colours for all census data (eg 50% for private motor vs rest, or for cycle to work rates where we’re so far behind Cambridge that we could do with adjusting the scale/buckets)
Forgive me if I’ve missed this: is there a reference to source tables for the Census tables used? For the Journey to work table, I expect that you’ve used WU03UK, but it’s not at all clear…
Hi Tim, it’s WU03EW as that has the data at MSOA level – WU03UK only is at LA level I think as Scotland haven’t released the more detailed information yet. & yes I’ve forgotten to mention this in the blog post about it so thanks, I’ve added a note to the blog now.
Excellent work, very useful.
A few minor GUI tweaks would make it even better:
1) allow toggling of placenames. Currently one name unavoidably completely obscures my home village, unless zoomed in so far that the overall visible area is too limited to be really helpful.
2) “Space equally” should be sticky so as to allow easy comparison between subranges of data, e.g. different ages
3) Allow windowing of the colour scales overlay (at the bottom) and the “datashine” overlay at top left – as is already done with the “key” and “data chooser” overlays. Try using it on a netbook to see why they take up too much screen area!
Hi Tom
1 – We have now made a change which allows this. Click the “Labels” button to remove the labels. This also extends to the PDF generator.
2 – We will look into this, although implementation of this particular feature may be tricky.
3 – We’ve tried to optimise this for small screens (e.g. iPad) but realise that it is still not ideal. It is a balance between showing a useful UI and not overwhelming the map itself. We could potentially create alternative stylesheets for smaller windows, that show a subset of features or smaller fonts, so we will investigate this.
I’d thought about 3 previously too – I think Tom’s suggestion is to make those windows collapse-able, as per this rough mock up: http://www.gmcc.org.uk/wp-content/uploads/2014/07/datashine.jpg
I’ve since realised that moving them around may have implications, but they’d work there at 1920×1080, and collapse-able in current locations would still be an improvement.
FYI it’s very usable as-is on a Samsung S5 but being able to collapse those two boxes would help.
ps A postcode search and the city links would be useful on the commute map too.
Excellent work, and lovely interface. Saves having to mess about in Mapinfo.
A quick thought – the census questions were to households, yet your building layer includes commercial buildings. This is slightly misleading on the mapping, as large effectively non-domestic Output Areas are rendered according to the local housing. This is particularly relevant in urban centres and industrial estates. Is there any way of excluding non-domestic buildings from your base layer?
Ah – i see you have already addressed this on 3rd July. As the last comment to that question states, this is still a big improvement, though would be even better if non-domestic properties could be removed (I realise this is out of your hands)
Hi Oliver,
I was wondering what kind of usage of this data you were happy with? I can’t find anywhere that seems to prohibit companies using the mapping in for profit work but obviously wanted to check before we looked into using it for any projects.
Thanks for any help,
Kat
Hi Kat,
The data themselves are from the census and so can be downloaded and used in accordance with the ONS’ Ts+Cs. Under step 2 of the “Data Chooser” box we have a “Data Download from NOMIS” link. Click on that for the data you need. If you want to use the maps, feel free to as long as we get full attribution. Thanks! James
Hi Oliver,
Great work! Really useful datasets and visualisation.
I’m writing to ask if there is any way I can get a bulk download of the datasets (layers and description of calculations used) for Rhondda Cynon Taf. I am a student at the University of South Wales studying MSc in Disaster Management for Environmental Hazards and involved in a research project for the Rhondda Cynon Taff Council Emergency Planning and Resilience unit. My dissertation entails mapping social vulnerability in RCT from an emergency planning perspective using ArcGIS. As part of my research I will require datasets similar to the ones you have created for datashine. The data that will be collated will be used to develop a system for assisting in planning for an emergency response situation.
Please advise as to whether this is something you can help me with.
Kind regards,
Martyn
Oliver,
Love to talk more about this.
Blogged over the weekend re application of such mapping in a local ad context…
http://rickwaghorn.co.uk/2014/07/12/with-a-perfect-sense-of-timing-microsofts-chief-envisioning-officer-sees-humans-on-the-rise-again-in-the-field-of-local-ad-sales-big-data-is-merely-another-tool-for-a-story-teller/
But there is a large opportunity looming here.
Best
Really interesting tool – great to see something that might get the people talking about Census data.
Can you tell me why some areas appear grey? I assume it’s not due to lack of data, since the Census should have covered the whole of England and Wales.
HI – yes, all areas in England/Wales should be covered. If the grey area looks square, it’s possibly just that that image tile was slow to load – try zooming out and back in. Also, note that Scotland and NI are currently shown on the map, but in grey.
What a beautiful tool to visualize data! I think this is fabulous.
I have a question about low percentage numbers.
How many persons do the percentage numbers refer to on the largest scale [200 m scale]?
And do the areas of the average values overlap?
Thanks very much. We use two geographical units for DataShine, at the moment. For smaller scales we use Wards (population of each around 7500) and for larger scales, such as the one with 200m bar, we use Output Areas which have a typical population of around 250. So 1% = 2 or 3 people. Note that the ONS may perturb some small numbers for reasons of confidentially/disclosure control – the idea being it should not be possible to identify individual people from aggregate statistics.
There should be no overlap in how the values are calculated – each OA (or Ward) has its percentage calculated independently. We do however look at values across every OA/Ward to calculate the average and standard deviation, which informs which type of key we use.
Great tool. Is there a way to highlight/augment the maps? For example when looking at cycle to work, it would be useful to be able to show high quality protected cycle routes on the map.
Hi Adam, unfortunately off-road cycle routes are not included in the OS Open Data datasets which we are using for our contextual map. OpenStreetMap data is a potential alternative datasource although it is a little involved to combine the two together.
Very cluttered map,
Viewing on a 10″ tablet is very difficult.
Pls make some dialogue boxes float, ie I want to move or minimise them, like in modern software program’s.
Thanks Chris. We tried hard to keep it as clean as possible given all the variables we wanted to display. Boxes with blue tops have a collapse option in their top left and they also float. HTH.
Hi
Wonderful work so far.
I wish to ask if i can somehow view two variables at a time.
e.g. If i can view religion as well as employment?
Thanks
Sajid
Hi Sajid,
Yes we think this would be useful too and plan to work on it for future releases.
Very impressive and useful site. Would like to be able to enter a comma separated list of postcodes, and see these appear as pins on the map. I wonder if that is something being considered???
Thanks you for making this available.
HI Andy, the current treatment of the postcodes is very simplistic – it actually just zooms to the centroid for the post “outcode”, that is, the bit before the space. The benefit of this is that the locations file is quite small, and we don’t need to use a database for it. If we wanted to use actual postcode locations, that would require quite a bit of re-architecting of the system.
This is a brilliant idea, well done! Although it’s a wee bit disappointing that there doesn’t appear to be any data available for Scotland or Northern Ireland – hopefully that will be available in due course 🙂
Hi Kirsty, yes I am hoping to add the Scotland data in due course – likely will be later this autumn. NI will be much trickier – not because of the data itself but because the lack of OS Open Data makes it hard to produce a similar looking map for NI. However, I will have a go!
I’d be really interested to know the various sources of data you use to build this. I have some ideas of my own…
Hi Sara. The demographic data comes from the Office of National Statistics – it’s their “Quick Statistics” releases of aggregated results for the 2011 census, at Output Area and Ward level, as collated by NOMIS. The mapping data comes from Ordnance Survey Open Data datasets, namely Vector Map District and Meridian 2. There’s more detail about this at http://blog.datashine.org.uk/2014/06/datashine-maps/
DataShine is intended to be a “platform” rather than just a single site, and will be using it to build further maps like this as the project (BODMAS) moves forward over the next few months.
It’s a fascinating tool. One question/criticism – is there a way to turn off the place names? For instance, if you zoom out as much as possible, so you’ve more or less got the whole country in view, the word ‘Manchester’ obscures not only the whole of Manchester, but most of Liverpool and even some of Sheffield too.
Hi Barney. I agree, and I’ll add a button to do just this (I need to rejig the labels in general, because yes they are too large at certain zoom levels, this will happen in the medium term.) Until the button is available, you can manually turn off the labels by changing the third “T” in the layers bit of the URL, to “F”. You may need to then hit enter a couple of times in the URL bar, due to the way some browsers handle “anchor” permalinks.
Thanks Oliver – though it seems to be the 2nd ‘T’ that turns off the names for me (So I use ‘BTFT’ rather than ‘BTTT’).
Thanks – I was guessing which one it was as the site was a bit overloaded at the time.
There is now a “Labels” button that toggles on/off the labels.
Excellent; I was just about to ask you to do this!
Overall it is a very interesting and useful site.
Hi
On an iPad, there’s so much of the screen given over to selection menus, various cities and comment/about that the map can’t be analysed
Hi Keith – we tested this on an iPad here and it looked OK. It may be a resolution thing – do you have an older iPad with lower resolution. Also, does it look better if you view it in landscape (i.e. tilted on its side)?
Also, if you click/touch the title bars for the blue widgets, they should collapse which should free up more of the screen.
Very useful and interesting map. I would find it interesting to see the election/referendum results and participation levels. E.g. I found it interesting to see the distributions of Yes votes on the AV referendum to suggest a correlation of education and voting preference (http://www.theguardian.com/news/datablog/interactive/2011/may/06/av-referendum-results-map)
Hi. It’s an issue that non-residential buildings (e.g. schools, offices) are shown shaded, and this misrepresentation may have unfortunate consequences. Hard to do much about it though undless the mapping encodes building use.
Yes, this is a big flaw with this kind of mapping. The dataset we use for the buildings (OS Open Data Vector Map District) doesn’t include any metadata telling us what type of building it is, so we cannot automatically distinguish. The other thing is to note that the data isn’t so granular that it differentiates between every house. Each areal unit (Output Area) typically covers around 150 addresses.
It is nevertheless an improvement on the standard choropleth visualisations where the eye is drawn to the larger blocks of colour.
Great maps. In Wales, you’ve lumped English and Welsh together as a main language, so we can’t see where Welsh is spoken the most. Any chance of adding a distinct Welsh tab?
Hi Luke, I think is available in a separate Welsh-only table from the ONS, which we haven’t included at this this stage. We’ve only included tables which have data for both England and Wales, for this first version. We hope to add additional tables in due course, including Scotland and the Welsh-only ones.
Looks really excellent – well done.
very minor issues:
Northern Ireland is not surrounded by sea.
There is a an island in the irish sea, where a famous motorbike race is held.
Not sure if stats were displaying for the Isles of Scilly.
Hi Rory, yes, good point, we need to include additional country polygons for non-UK. It’s on the TODO list!
The Isles of Scilly should be on there.
Hi, I see huge potential in this. Are you planning in releasing the structured supporting data along with the mapping detail as part of a area look up.
Hi Aaron – yes, one of our next planned improvements is to have a local area view (with appropriate colours), and as part of that, we hope to provide a data download for the local view too. Watch this space and sign up for our newsletter, as we will likely announce it there first when it is ready!
Very good indeed. A fantastic way to explore the UK and to see which bits are “nice” vs “nasty”
Brilliant map and presentation of the data.
What’s the underlying basis? OSM?
Hi, the underlying/contextual data is Ordnance Survey Open Data. We didn’t use OSM as its building coverage (which is crucial for the kind of map here) is still quite incomplete. However if/when we extent DataShine Census to Northern lreland, we will very likely be using OpenStreeetMap (OSM) data, as OS Open Data is not available for there.
Indeed, the OSM buildings layer is strongly biased towards areas with a high geek population like Cambridge.
Very useful tool to use when discussing with LA’s levels of Adult Cycle Training, thanks
Excellent tool and very good interface. I do local Neighborhood Planning, your tool could be invaluable in supporting community Neighborhood Forums across England and especially in urban areas for their evidence base reporting requirements. What is needed is to be able to draw their Neighborhood Boundary Area (which is often not Ward based) and to see the aggregate data count view for their Neighborhood Area creating a dashboard of key information. eg total household for the area, total population (may be also showing the electoral roll counts for the area too), age breadown of age etc.
Minor comment, on the Key (Locked, Release, Space Equally), can we have one for a defined increment? Presently, if you select Method of Travel to Work as Driving a Car or Van, then the lowerst % band is 0-12.5% and that is too big a catchment band to be useful for assessing Inner and Outer London profiling.
Thanks Francis – glad you like it.
We can look into that – its always hard to balance the priorities of “expert” users looking to use the tool in anger with those who want to just check out their local area etc. The project will be developing some more advanced functionality so I think there is the potential to wrap up manual colour breaks into that.
Very nice product: any chance of an option to include LA boundaries?
Hi Martin, LA boundaries would indeed be an obvious thing to include, so we’ll look to put these in in a subsequent update.
Very interesting/useful stuff. Im only just trying to get my head around it. However one thing that would be very useful is the accumulated data for say. one town, region, county etc. Is this possible?
glad you like it. It may be possible and probably something we would look at doing in a future release.
Awesome map…would be nice to remove the transparency on the dialogs though…
Hi – yes, it’s difficult to get the balance right, we don’t want the dialogs to be difficult to read due the detail underneath, however we do still want to have the sensation of the map covering the full area of the screen, and the dialogs “floating” on top of it. We will look to making the dialogs slightly more opaque in a future update.
Very useful indeed. Have you uploaded the comparable statistics from the 2001 or earlier censuses? That would be useful for seeing how some areas and communities change over time.
Hi Therese – There is some similar work we did for 2001 here – http://censusprofiler.org/ – although we don’t have any plans yet to add in earlier censuses, this is certainly a possibility for future work.
It’d be useful to see the travel-to-work data with not-in-employment and work-at-home excluded. In particular, the uneven geographical distribution of not-in-employment makes a huge difference (certainly in the likes of Oxford).
Yes, Travel to Work is a particularly interesting dataset and potentially we could be building a specialised site to show it in detail, in the future, and with exclusions like those who don’t travel at all to work. For now, it does include all residents aged 16-74.
I’d also like to see ‘total public transport’ (train/bus/minibus/underground/tram etc.) and ‘total private transport’ (driving/passenger in car/van, bicycle etc.) options for Travel to Work. However, I’m not sure which category – if either – ‘taxi’ would fit into!
At the moment we are simply presenting the ONS aggregate table breakdown “as is” rather than combining/splitting the tables ourselves. I agree it would be a useful aggregation. Should we do a dedicated Travel to Work map in the future then it would likely have more helpful aggregations like these you suggest.
I support both these requests, but wouldn’t group cycling or walking with non-sustainable private modes.
The destination info in the commute map is great, however it’d be useful to have totals by mode for a destination, ie choose mode: “All (totals only, no flows)” then don’t show the flow lines and change the data box to list totals (inbound) by mode. Even better if it can include routes with <6 people and travel within the area too.
This resource has proved useful already and I'm sure it's got lots more to offer – please keep up your excellent work!