Unreal Estate

At the Haggerston West Estate in Hackney, East London, giant heads stare out from the windows. These are portraits of the remaining residents. Many of the other windows are boarded up with orange panels, which started to be installed five years ago when this building was condemned. A group of artists called Fugitive Images, whose members are among those still living here, decided to battle these symbols of dereliction with signs of life. The message is spelled out across three groundfloor windows: I Am Here.

The building next door has already gone. The construction hoarding around this void is advertising a "luxury development" called Haggerston Square. The artist's impression shows a series of large blocks with multicoloured facades. "Coming Soon," it says.

There's been a lot of this recently. Across Hackney, from Haggerston to Dalston and Homerton, luxury development is on the march. You wonder what criteria developers need to satisfy to qualify for that term "luxury". Is it the use of plasticky facade panels? But more worrying is what they are replacing, which in some cases are council estates like the one with the defiant portraits. Built in the 1930s — on a block that Charles Booth's Poverty Map had classified as "vicious, semi-criminal" — these "neo-Georgian flatted dwellings" are now in the middle of one of the most competitive development zones in London. It will apparently be replaced by a "lively", "vibrant", "diverse" community. "Why are these communities never seen that way before the developers come in?" asks Andrea Luka Zimmerman of Fugitive Images, skewering the marketing-brochure patter. "This is language as an empty signifier."

Across London, council housing is out and the luxury development is in. This is the endgame of a process that began 30 years ago, when Thatcher pulled the plug on state-built social housing and gave millions of people renting it the "right to buy". Londoners are innately aware of the effect those two policies have had on their lives. An acute housing shortage along with the privatisation of social housing has made it near impossible not just for lower-income but also middle-class people to own property anywhere near the city centre. The housing crisis has been getting worse since the recession, with housebuilders building less and banks not lending. Rents rose 12 per cent last year, waiting lists are getting longer and homelessness is going up.

"You could get lost in this concrete jungle / new builds keep springing up outta nowhere," sings rapper Plan B in his recent single Ill Manors, an angry jab at the social conditions that led to last year's riots. The video was shot in the Forest Gate area of Newham, in East London, one of the poorest boroughs in the country. Here and in other boroughs in and around the London Olympics site, thousands of new units are being built in one of the biggest regeneration projects the city has ever seen. You wouldn't know there was a housing crisis on. But then Londoners are used to these paradoxes. It rained almost every day in April and the papers were full of headlines about drought. The water table is low apparently. Well, despite all the new builds, the housing table is low too, from decades' worth of undersupply.

In the postwar years of the welfare state consensus local authorities were building hundreds of thousands of homes a year in England. That rate of production was relatively constant until the late 1970s, regardless of which party was in power. But if you look at a graph of social housing production you'll see it drop sharply from 1979, the year Thatcher took office, and fall steadily through subsequent governments to almost zero — a trend that New Labour's policy of forcing developers to include some affordable housing did very little to reverse. Thatcher believed that the private sector should and would step in where the government had stepped away — creating a nation of homeowning Tory voters. But her faith in the market proved very wide of the mark. Housebuilders have been building roughly the same number of houses per year since the mid-1950s. There was no rushing into the breach. Which means that either they are building at capacity or there is simply no incentive for them to produce more. After all, why would they devalue their product by lowering the demand?

On a Saturday in Bow, there are dozens of groups setting off on guided tours of the Olympic site. "Let's all go on an urban safari," goes Ill Manors. From here at Three Mills Green, south of the Olympic Park, you can already see a new landscape taking shape in the distance. The Spirit of Stratford, a 43-storey residential tower in the freshly regenerated Stratford City, is the new beacon in this part of town. With its rainbow facade, like a vertical case of Caran d'Ache colouring pencils, it's a developer's wet dream. It's also the sort of highrise that was supposed to be unpopular in this country, discredited architecturally and politically, at least when it was called council housing. But this hulking, neoliberal variant is apparently acceptable, not least because it is helping Newham to meet its housing targets — 655 units of "legacy" right here. The fact that flats in the Spirit of Stratford are selling like hot cakes to buy-to-let customers in China, however, suggests that it may not help solve the borough's housing crisis.

Newham sparked a political scandal recently when it emerged that it was having to house hundreds of families in Stoke-on-Trent, 160 miles from London. With the Coalition government's cap on housing benefits, the borough said it simply couldn't afford to house these residents locally. Critics of the Coalition have been predicting that its austerity policies would see a mass exodus of the poor from the capital. Even the Tory Mayor of London Boris Johnson distanced himself from the government on this one, claiming he wouldn't allow "Kosovo-style social cleansing" in his city. The mayor is prone to hyperbolic rhetoric, but on this occasion he was perhaps a little too vivid for his own good, allowing the most damning critique of 21st-century London yet uttered to fall from his own lips.

And that's not all. The news has been full of reports that Newham is cultivating a new breed of secret slum. In the back alleys between terraced houses, landlords have been erecting flimsy extensions — "super-sheds" the papers call them — crowded full of immigrant workers paying hundreds of pounds a week. "It doesn't take long for you to make a lot of money out of it, provided you are prepared to trade in human misery," Newham's mayor Robin Wales told the Guardian. So here at last are the Euro-slums, the return of the Dickensian rookeries. If they didn't exist, London would have to invent them. A special taskforce is charged with rooting out the super-sheds using helicopters and aerial photography. Meanwhile, only a mile or so away, the shimmering new Stratford City and its Westfield mega-mall promise to serve a new, upwardly mobile community.

Just across the newly cleaned River Lea, Bow is home to more phenomena that sound like urban myths but happen to be true. Residents are jittery over plans to install anti-aircraft missiles on the roof of an apartment building here to prevent a 9/11-style terrorist attack during the Olympics.

Less sensational but more interesting in the long run is that, nearby, Ikea is building a whole neighbourhood of 1,200 homes. Strand East, as this 26-hectare former industrial site is called, has had the pundits reaching for jokes about flat-pack housing. There is little to see at this point apart from some bulldozed land, a few preserved warehouses and a giant wooden tower modelled on the Olympic Torch. Though CABE, the architecture watchdog, has criticised the limited amount of public space in Ikea's proposal, the early plans suggest that Strand East will be considerably more sensitive than the vision perpetrated in Stratford. Which raises the possibility that even this unexpected corner of the private sector understands city-making better than many developers.

The same is probably not true of supermarket giant Tesco, which plans to build its own little town just down the river. What sits there now is the Bromley-by-Bow Tesco superstore, a fading suburban model sandwiched between the A12 and its own vast parking lot. The supermarket aims to build around 300 homes here, divided between a mid-rise city block and a glass tower, neither of which look like they have anything to do with the other. Only 76 are slated as "affordable". Affordable housing is what we have instead of social housing, and the Coalition government has decreed that affordable rents can now be up to 80 per cent of market value, which means that they are hardly for the poorest. But the real question over both the Ikea and Tesco schemes is whether they offer anything new. Will there be insidious branding, product placement, public-private ambiguity and all the trappings of an Ikea or Tesco Merzbau? In London, letting global retail corporations build entire communities counts as innovation. And the floodgates are now open. If McDonalds, an official sponsor of the Olympics, decided it wanted to go into housing in East London, the path would be clear. Luckily, it decided to build the biggest restaurant in the world here instead.

Dispiriting though much of this is, it would be churlish not to mention that East London is also home to some enlightened housing projects of recent years. Staying in Bow, there is Peter Barber's Donnybrook Quarter, a mix of terraced and courtyard houses like a North African kasbah reimagined. A few miles east of the Olympic Park, there is the redevelopment of Barking town centre by ahmm and muf, where new flats and amenities are arranged around a truly thoughtful public square. Down the road is ahmm and Maccreanor Lavington's Anne Mews, two rows of generous brick terraces, just across the tracks from another Peter Barber project, Tanner Street Gateway. These are all-too-rare cases of housing that delivers more than the bare minimum, that has civic ambition.

And Anne Mews is even more rare for being council housing rather than merely providing a percentage of affordable homes. The chances of projects like this being delivered again in the near future are slim, now that the mayor has cut the funding for housing associations by more than 70 per cent. Where is the extra money to come from? The private sector of course.

London is perpetually redefining the extremes of what the private sector can deliver. If it can produce super-shed slums, it can also lead the world in ludicrously expensive pieds-à-terre. Take One Hyde Park in Knightsbridge, a development designed by Rogers Stirk Harbour + Partners for the Candy brothers. The ground floor hosts a branch of the Abu Dhabi Islamic Bank, while somewhere above that a flat sold last year for 135 million pounds. Needless to say, there is no "affordable housing" quotient — at 80 per cent rents even your average mining magnate would struggle — but that's because the developers of such elite cribs pay the council to build the social units somewhere else.

One of the problems of the capital's property market — for locals that is — is that it's such a safe investment. London is a haven for fleeing capital. A wave of political and economic crises abroad has brought money flooding into the property market recently from Libya and Egypt, Italy and Greece. But beyond the security of its bricks and mortar, London is also the capital of a de facto tax haven for the super-rich. The fact that so-called non-domiciles — or "non-doms" — flock here makes life more difficult for the rest of us. The UK government courts them because it believes in the "trickle down" effect of their enormous wealth. But not only is this notion now thoroughly discredited, there is, as David Harvey tells us, no spatial version of "trickle down". More expensive flats for the rich do not lead to more flats for the poor — quite the opposite. "The idea that a city can do well (in terms of capital accumulation) while its people (apart from a privileged class)… do badly, is never examined," says Harvey.

At the Heygate Estate in Elephant and Castle, in South London, 1,260 flats are boarded up and awaiting demolition. These giant slabs of municipal modernism are victims of a Thatcherite political agenda that successfully vilified council housing for its attendant social problems, especially if it was on an industrial scale and smacked of mid-century architectural utopianism. The rhetoric of "failed estates" — or "ill manors" as Plan B calls them — held architects responsible, deploying gifts such as the collapse of the Ronan Point tower in 1968 rather than a failure to invest in maintenance.

You can still stroll Heygate's elevated walkways overlooking its generous gardens. Predictably, around the jungle gym someone's shooting a pop video. Here and in Haggerston, it is the final ignominy of council housing that it should serve only as a gritty stage set or as a support for art. But if this defunct future is picturesquely depressing, the new future is just demoralising. It comes with barcode facades and wind turbines purely for show, like the Strata tower right next door.

Replacing Heygate will be a new development of 2,500 units, so twice as dense, but only 25 per cent will be "affordable". Former mayor Ken Livingstone had managed to get the affordable housing quota up to 50 per cent of new developments, but Johnson scrapped that policy in favour of allowing boroughs to set their own targets. So only about half of the original residents will now be able to return, and only if they can afford it. The residents of Haggerston are luckier, since all of them will be rehoused in the new development, "pepperpotted", as the jargon has it, with wealthier private owners. Whatever your views on mixed-income housing — whether you see it as gentrification or social diversity — what the destruction of these estates represents is the end of social housing as a zone of exception, free from the logic of the market. Politically, of course, this is old news. What Heygate represents is merely the erasure of the evidence. To defend a Heygate — or, more popularly, a Robin Hood Gardens — is decried as fetishism.

Which may be justified if the fetish is for a moment when architects still had political agency, but not if it's for the remnants of a progressive politics. The population of London is projected to grow from 7.9 million to 9 million by 2030, which means we need to build 32,000 homes a year to keep up with demand. We're building about half that. The problem with relying on the private sector to deliver those homes is not just that too few of them fall into the hands of the needy. There's a fundamental economic disincentive, since there is just as much money to be made speculating on and refinancing the existing output as there is in producing more. Harvey again: "Stimulating demand by taxation and public policy gimmicks and other incentives (such as increasing the volume of sub-prime mortgages) does not necessarily elicit an increased supply: it merely inflates prices and stimulates speculation."

If it is not yet clear after 30 years of grinding those wheels that the market cannot solve the housing crisis, then I'm not sure when it will be. Similarly, it is obvious to anyone who lives in this city that the market does not operate in the interests of those on lower incomes. So why do we keep slavishly turning to the market? It is a hopeless moment to make this argument, with the government slashing funds for social-rented housing across the country, but it's time for the public sector to resume its responsibilities. It's not just the only way for the government to meet its beloved targets; it's the only way to re-establish the zones of exception that protect ordinary citizens from a ruthless market. Justin Mcguirk (@justinmcguirk)