23 June 2018

How backups, backups, backups protect NYC’s cellular infrastructure


The infrastructure that underpins our lives is not something we ever want to think about. Nothing good has come from suddenly needing to wonder “where does my water come from?” or “how does electricity connect into my home?” That pondering gets even more intense when we talk about cellular infrastructure, where a single dropped call or a choppy YouTube video can cause an expletive-laden tirade.

Recently, I visited Verizon’s cellular switch for the New York City metro area (disclosure: TechCrunch is owned by Oath, and Oath is part of Verizon). It’s a completely nondescript building in a nondescript suburb north of the city, so nondescript that it took Verizon’s representative about 15 minutes of circling around just to find it (frankly, the best security through obscurity I have seen in some time).

This switch, along with its sister, powers all cellular service in New York City, including three million voice or voice over LTE (VoLTE) calls and 708 million data connections a day. High-reliability and redundancy is a must for the facility, where dropping even one in 100,000 connections would create more than 7,000 angry customers a day. As Christine Williams, the senior operations manager who oversees the facility, explained, “It doesn’t matter what percentage of dropped calls you have if you are that person.”

As we walked through the server rows that processed those hundreds of millions of connections, I was surprised by just how little digital equipment was actually in the switch itself. “Software-defined networking” has taken full hold here, according to Michele White, who is Verizon’s Executive Director for Network Assurance in the U.S. northeast. As the team has replaced older equipment, the actual physical footprint has continued to downsize, even today. All of New York City’s traffic is run from a handful of feet of server racks.

The key to network assurance is two-fold. First is multiple levels of redundancy at every level of the infrastructure. Inside the switch, independent server racks can take over from other servers that fail, providing redundancy at the machine level. If the air conditioning — which is critical for machine performance — were to fail, mobile AC units can be deployed to pick up the burden.

All equipment in the building is serviced by DC power, and in the event of an external power loss, two diesel generators connected to a large fuel storage tank will take over. The facility is also equipped with battery backups that can sustain the facility for eight hours if the generators themselves don’t function appropriately.

Diesel generators can sustain power to the switch in the event of an external power outage

At a higher level, the switch and its sister share all New York City cellular traffic, but either one could handle the full load if necessary. In short, the goal of the switch’s design is to ensure that that no matter how small or large a problem it might experience, there is an instant backup ready to go to keep those cellular connections alive.

The other half of network assurance is centralization, something that I was surprised to hear in this supposed era of decentralization. Cellular sites in an urban area like New York are often placed on buildings, as anyone looking at roof lines can see from the street. Given those locations, it can be hard to provide backup generators and other failover infrastructure, and servicing them can also be challenging. With centralization, increasingly only the antenna is located at the site, with almost all other operations handled in central control offices and switches where Verizon has greater control of the environment.

Even with intense focus on redundancy, natural disasters can overwhelm even the best laid plans. The telecom company has an additional layer of redundancy with its mobile units, which are placed in a “barnyard” owing to the names of the equipment stored there. There are GOATs (generator on a truck), and COWs (cell on wheels), and BATs (bi-directional amplifier on a truck). These units get deployed to areas of the network that either are experiencing unusually strong demand (think the U.S. Open or a presidential inauguration) or where a natural disaster has stuck (like Hurricane Harvey).

A barnyard filled with animal-named mobile cell infrastructure, including COWs, COLTs, HORSEs, and others

That said, both White and Williams noted that mobile cell deployment is much rarer than people would guess. One reason is that cell sites are increasingly being installed with Remote Electrical Tilt, which allows nearby cell sites to adjust their antennas so as to provide some signal to an area formerly covered by an out-of-commission cell. That process I was told is increasingly automated, allowing the network to essentially self-heal itself in emergencies.

The other reason their deployment is rare is that network assurance already has to handle a remarkable amount of surging traffic throughout the normal ebb and flow of a dense urban city. “Rush hour in Times Square is pretty heavy,” noted Williams. Even something as heavy as a parade through Midtown Manhattan won’t typically exceed the network’s surge capacity.

One other redundancy that Verizon has been exploring is using drones to provide more adaptive coverage. The company has been testing “femto-cell” drone aircraft designed by American Aerospace Technologies that can provide one square mile of coverage for about sixteen hours. A drone capability could be particularly useful in cases like hurricanes, where roads are often littered with debris, making it hard for network engineers to deploy ground-based mobile cells.

I asked about 5G, which I have been covering more heavily this year as telecom deployments pick up. Given the current design of 5G, White and Williams didn’t expect too much change to happen at the switch level, where most of the core technology was likely to remain unchanged.

The trend that is changing things though is edge computing, which is in vogue due to the need for computing to be located closer to users to power applications like virtual reality and autonomous cars. That’s critical, because 50 milliseconds of extra latency could be the difference between an autonomous car hitting another vehicle or a new support pylon and swerving out of the way just in time.

Edge computing in many ways is decentralizing, and therefore there is a tension with the increasingly centralized nature of mobile communications infrastructure. Switches like this one are getting outfitted with edge technology, and more installations are expected in the coming years. 5G and edge are also deeply connected at the antenna level, and that will likely affect cell deployments far more than the switch infrastructure itself.

Edge, internet of things, 5G — all will increase the quantity and scale of the connections flowing through these networks. In the future, a cellular outage may not just inconvenience that YouTube user, but could also prevent an automobile from successfully navigating to a hospital during a natural disaster. It takes backups, backups, and backups to prevent us from ever having to ask, “where does that signal come from?”


Read Full Article

YouTube Adds New Ways for YouTubers to Make Money


YouTubers are the lifeblood of YouTube. Without the people creating content, YouTube would cease to be. Unfortunately, YouTube has not always remembered that, and the so-called Adpocalypse of 2017 led to bad blood between YouTube and its content creators.

However, YouTube has now unveiled a trio of new ways for YouTubers to make money. And together they represent YouTube looking beyond advertising revenue. Which, given that YouTube is owned by Google, one of the online advertising giants, is important.

YouTube’s New Revenue Streams

In a post on the YouTube Blog, Chief Product Officer, Neal Mohan, explains how YouTube now boasts almost 2 billion users every month. However, YouTube wants content creators to be able to earn a serious crust, which is why YouTube is looking beyond advertising.

Channel Memberships are an extension of the existing Sponsorships. Viewers pay $4.99/month to a content creator, and in return get unique badges, new emoji, members-only posts, and “access to unique custom perks offered by creators”.

YouTube is initially limiting Channel Memberships to “eligible channels with more than 100,000 subscribers,” which means you need to be popular before you can cash in. However, YouTube suggests that these requirements may change in the future.

Lots of YouTubers sell merchandise to help bring in some extra cash. However, YouTube is now providing the means to do so directly on the platform. With Merchandise, YouTubers (based in the U.S.) will be able to customize 20 different items from Teespring.

Last but not least is Premieres, which enables YouTubers to debut pre-recorded video as a live moment. This means lots of viewers will be watching simultaneously, allowing creators to use Super Chat (and its tipping system) on traditional uploads for the first time.

YouTubers Bring Home the Bacon

These are all solid ways for YouTubers to bring in extra revenue, and a much-needed shot in the arm for YouTube’s relationship with its partners. And for the rest of us, who merely watch YouTube videos, these should help us connect to the YouTubers we follow.

It should be noted that YouTube isn’t the only game in town, even though everyone is still using YouTube. However, if you have already tried some of the YouTube alternatives before returning to the fold, here are the monetization tactics used by the pros.

Image Credit: Rego Korosi/Flickr

Read the full article: YouTube Adds New Ways for YouTubers to Make Money


Read Full Article