60 Minutes eviscerates Boeing

 - 
01/11/2010
Whatever you think of 60 Minutes (I happen to think they're a damn fine collection of journalists, though Andy Rooney should hang 'em up at some point in the very near future), their report last night on SBInet is a real black eye for the security community. The project, famously won by Boeing in 2006, promised a "virtual fence" stretching the entirety of the border with Mexcio. Boeing promised the entire 2,000 miles of it would be covered by now. Instead, there's about 28 miles of it covered, and that part relatively ineffectively. The cynical among us might say that's hardly news. Boeing might get defense and make some awesome rockets, but they're not security guys, and as Steve Kroft says, there in some ways isn't much difference between your standard commercial alarm system and what they want to do on the border. It's intrusion detection, first and foremost, and for intrusion you need security guys. Monitoring guys. To tell the truth, I bet they could do with a real nice Sonitrol system extending the length of the border. They'd hear them coming and know right where they are. Boeing doesn't seem to have a clue? That's what you were all telling me from the time they go the bid, but to see it laid out in this fashion is particularly embarrassing for someone like myself who finds himself defending the industry in polite conversation more often than I'd like. Let's just say I'm glad this aired after Christmas dinner. Take a look for yourself and then I'll give you what I feel to be the lowlights: Ugly, right? I'd say the executive director of the operation, Mark Borkowski, is the star of the show. Rarely do you see someone in government as candid as he is, even if he's basically throwing Boeing under the bus most of the time. Most damning, for me, is the arrogance shown by Boeing - and everyone involved, really - by not working with the customer, the end user, on designing the system. Has no one at Boeing learned a single thing about the basics of sales and marketing? I love the exchange where Steve Kroft asks, "Is this an Edsel?" and Richard Stanna, the GAO administrator charged with examining SBInet replies, "we're waiting for something that works." He doesn't even know if it's an Edsel yet because he still can't even really take the thing for a test drive. Boeing promised the government it would have 2,000 miles of surveillance (video-analytic based, radar integrated, outdoor surveillance, no less) installed in three years? No one there has been introduced to the old "under-promise/over-deliver maxim" I'm guessing. Sure, they're under pressure to get the bid - nobody sneezes at $1 billion in revenue - but, as Borkowski rightly says, "shame on us," shame on the government administrators, for believing that promise. Also, I really can't fathom with what's meant by Borkowski with his comments about COTS equipment being used? Really? What counts as COTS, is the question, I guess. Are Pelco cameras COTS? Are they using consumer-grade stuff? I'd definitely like to know more there. The government's definition of COTS is probably a lot different than what we use in the commercial integration biz. $1 billion in and we've got 28 miles that, by Stanna's estimation, "sorta works." That ought to help security's image...

Comments

At this point, it is clear that video analytics 1.0 dead and knee jerk spending on homeland security systems are wasteful. Having this showcased in a major news program reinforces this for the general public.

I do not think it matters who did this project as the sheer nature of the task is beyond current technological capability. Boeing, ADT, JCI, etc. would all face similar nearly impossible obstacles to accurately detect intruders over such a vast and challenging terrain.

Agreed. I'll go even further and say that this is the kind of thing where the low tech solution works best. A bunch of thermal cameras and outdoor sensors using wireless mesh to connect to a control center, staffed by humans, combined with several quick reaction forces equiped with iPhones (for communication and for viewing the cameras; FLIR + Firetide + Exacq + Apple/ AT&T = COTS!) and Hummers, and this thing could have been operational 18 months ago.

Yes, it would be operational but it would have abandoned the pretense/objective of using systems to automatically identify intruders. Maybe that's the right and realistic approach to take but it would be radically different and limited compared to their 'concept.'

I think it's the responsibility of the integrator to explain the concept of "can't be done with the technology available on planet Earth in the year 2007" to the project manager. If you accept their money for something you know you can never deliver, shame on you. Boeing should give back any money they've accepted so far.

As someone who's sold the Federal government for a long time, let me reinforce that Boeing knows all about the basics of sales and marketing. In fact, in this case it's ALL they know.
What I mean is that the salesguy finds out there's a bid/project/initiative coming. He does everything he can to get the business which means 1) leveraging government sponsors (they have a legion of them) who helped get previous contracts, who can 2) put the project on a Boeing "track" within the purchasing agency. It really doesn't matter if Boeing is supplying tiddlywinks or B-52s as the solution--the key thing is to win the biz which has everything to do with symbiotic relationships and little to do with technical merit.
Once the deal is won, it gets handed to some under-qualfied group to figure out what to do.
I watched the Navy outsource its entire IT operation to EDS in six months. Why? Because it was better, cheaper, more efficient? Hardly: it was a disaster. It was because EDS and Cisco used it as a sales tactic to cut the competition out and get all the budget.

One billion dollars to cover a 28 mile stretch of land? 35.7 million per mile? Is it just me or does that seem a little excessive? I just do not understand how large corporations can get away with this lack of accountability. I know in my business if I make contractual obligation to meet either a deadline or a performance spec I am held accountable. Why on earth isn’t Boeing?

What is forgotten in the environmental conditions in that region. Most of the current digital wireless/surveillance equipment would not hold up in these conditions.
From 1997-2004, over two hundred sites were installed using analog microwave (7 and 15 GHz), military grade PTZ recievers (Betatech), Hitachi 1/2"/Cosmicar cameras, and Lockheed Martin LTC550/500 thermal imagining cameras. This stuff worked at temperatures where you could cook and egg on the enclosure.

We heard toward the end of the IMC contract that they wanted to put "IP addresses" on every platform. Buzz words for those that have no idea what this means. Digital radios in cabinets and underfunded border techs to provide the continuous maintanence that such a project needed. Devices such as analytic boxes at the sites may have made this more advanced but it just plain worked. Analog microwave is the answer and those IMC Omega and Alpha radios were workhorses.

Enter the political bull where the goverment decides to buy a couple of hundred million dollars woth of a certain brand of thermal cameras (because someone did not like Lockheed or maybe they were not from the right state). These things were junk and never worked. Their fault but just happens that John Ashcroft visits the sector HQ and sees that night cameras not working. "Americans are fleeced" etc, etc, etc. L2 owned IMC by this point. The real problem is someone trying to cover their but and a government which was not interested in buying a service contract on the systems, thinking they could fix it themselves (even though they never did). Remember, you are asking wireless/surveillance equipment to operate in the desert at sometimes over 150F.

Another point is that from 1997-2001, about two hundred sites were installed (covering most of the southern border). After 9/11, and up to 2004, when the contract was lost, only a handful were installed because of the building of "Homeland Security". Why the momentum was lost was political and based on the interuption of funding during the agencies restructuring (why at this time of crisis was a crime to the US public)

Bring back the stuff that works.

I have to correct a few inaccuracies:

1-It is not accurate that Boeing was expected to finish the whole border project in 3 years. Boeing was paid for a first phase that included a test area of around 30 miles.
2- The whole discussion of analytics is irrelevant since i can tell you first hand that VA was not deployed for the testing period of the project. regardless, VA is indeed not the applicable fit, the way the solution is currently designed
3- The Mexicans already found a way around the security towers, they realized there is a deadzone right under the towers. As a San Diego resident, i can tell you that the fence is being penetrated every day and patched everyday right in front of the 80 ft towers.
4- The biggest problem that was not even discussed is, WHAT DO YOU DO IN CASE OF INTRUSION?? the current protocol allows intruders 90% of the time to run back to the Mexico side and disappear. no shooting is allowed, no non-lethal, not anything. so they try again the next day...
From my experience in the Israeli military, the border with Lebanon has a security system that had proved itself for 30 years. why reinvent the wheel at the cost of the TaxPayers money? 

Sagy, thanks for your comments:

1. I'm not sure that's better. So it was always going to cost $1 billion to secure an initial 30 miles? That was the plan?
2. Okay. See point 3.
3. So, if they're not using video analytics, why is there a "dead zone"? Dead zone from what? A dead zone from the cameras themselves? Because they can't put a camera pointing straight from the base of the tower? Dead zone from motion detectors or something? This makes no sense to me. And, what, you cruise out into the desert on a regular basis because you live in San Diego and check out the fences underneath the towers? Or are these towers right in the middle of the city or something? Or is it just on the news? I'm really not understanding the dead zone and fence breaching/repair.
4. Do you shoot them on the border with Lebanon? I'm not understanding.

Great article and message, only that the real let down is the government oversight not just Boeing.  This is the second colossal failure of the same project not first.  Boeing won what was substantially a re-compete of a project first failed by L-3 Communications along the southwest border.  I think it speaks more at the incompetence of project management capability internally in the government and the inability to oversee technical projects. 

Sam,

1-Ii was by no means trying to justify the cost or the timeline of this project. in fact, with the 50M Boeing got for the testing phase, I am sure I could have designed a better solution.
2-There is a dead zone under the towers because they did not consider this obvious issue. they have long range radar, and PTZ's with visible and IR sensors, but nothing for the shorter distance. While i do cruise out into the desert for fun, this specific cruise was in a BP vehicle while we tried to determine the possible use of VA. by the way, some of those towers are indeed in the middle of the city, or two city's to be accurate. Tijuana and San Diego. in some cases the houses on the Mexican side of the border are just a few feet away from the fence...

[...] a bad area to over-promise and under-deliver. I’m more alarmist as a taxpayer when I hear the 60 Minutes report on $1 billion spent on a 2000 mile “virtual fence” that only covers 28 miles and [...]

[...] month for major federal contractors tasked with deploying giant intrusion detection systems. First 60 Minutes made Boeing’s border project look very, very bad, now NYC’s ABC affiliate does a pretty good job of making Raytheon’s perimeter solution [...]

[...] is a bad area to over-promise and under-deliver. I’m more alarmist as a taxpayer when I hear the 60 Minutes report on $1 billion spent on a 2000 mile “virtual fence” that only covers 28 miles and “sorta [...]

[...] week I mentioned a $1 billion government contract that has only delivered a prototype that “sorta works.” When introducing new technology, it’s [...]

If 60 minutes is so uninformed about whta constittues COTS they should google it or check with their lawyers. It is codified by our lawmakers. That would be Congress.

Sorry to all of you but nowadays in 2010 there is at least on border control that was implemented succesfully:
http://ivcablog.wordpress.com/2009/08/30/belgian-made-video-analysis-rev...

[...] biggest challenges to our national security efforts. In case you didn’t get a chance to read Sam’s blog a couple weeks ago discussing a 60 Minutes episode on the ineffectiveness of the technology along [...]

[...] I wrote before, when 60 minutes eviscerated SBInet, this is a major black eye for the industry. And, in essence, it really doesn’t matter why [...]

[...] You might remember that 60 Minutes produced a great piece on the project not long ago (see Sam’s blog and the video here). [...]

[...] “virtual fence” was eliminated because, well, it didn’t work. There have been plenty of calls to action along the border, including the Texas Border Coalition [...]

[...] Initiative Network along the Mexican border, so you know things weren’t going well. A damning 60 Minutes piece also exposed the failures of the 28-mile pilot test that took three years to figure out it [...]