There’s a way to pick the absolute best images for your content: Apply AI

Kristin Tynski Contributor

Kristin Tynski is co-founder and SVP creative at Fractl, a growth marketing agency that’s helped Fortune 500 companies and boutique businesses earn quality media coverage, backlinks, awareness and authority.

Most marketers believe there’s a lot of value in having relevant, engaging images featured in content.

But selecting the “right” images for blog posts, social media posts or video thumbnails has historically been a subjective process. Social media and SEO gurus have a slew of advice on picking the right images, but this advice typically lacks real empirical data.

This got me thinking: Is there a data-driven — or even better, an AI-driven — process for gaining deeper insight into which images are more likely to perform well (aka more likely to garner human attention and sharing behavior)?

The technique for finding optimal photos

In July of 2019, a fascinating new machine learning paper called “Intrinsic Image Popularity Assessment” was published. This new model has found a reliable way to predict an image’s likely “popularity” (estimation of likelihood the image will get a like on Instagram).

It also showed an ability to outperform humans, with a 76.65% accuracy on predicting how many likes an Instagram photo would garner versus a human accuracy of 72.40%.

Image Credits: Ding/Ma/Wang (opens in a new window)

I used the model and source code from this paper to come up with how marketers can improve their chances of selecting images that will have the best impact on their content.

Finding the best screen caps to use for a video

One of the most important aspects of video optimization is the choice of the video’s thumbnail.

According to Google, 90% of the top performing videos on the platform use a custom selected image. Click-through rates, and ultimately view counts, can be greatly influenced by how eye-catching a video title and thumbnail are to a searcher,

In recent years, Google has applied AI to automate video thumbnail extraction, attempting to help users find thumbnails from their videos that are more likely to attract attention and click-throughs.

Unfortunately, with only three provided options to choose from, it’s unlikely the thumbnails Google currently recommends are the best thumbnails for any given video.

That’s where AI comes in.

With some simple code, it’s possible to run the “intrinsic popularity score” (as derived by a model similar to the one discussed in this article) against all of the individual frames of a video, providing a much wider range of options.

The code to do this is available here. This script downloads a YouTube video, splits it into frames as .jpg images, and runs the model on each image, providing a predicted popularity score for each frame image.
Caveat: It is important to remember that this model was trained and tested on Instagram images. Given the similarity in behavior for clicking on an Instagram photo or a YouTube thumbnail, we feel it’s likely (though never tested) that if a thumbnail is predicted to do well as an Instagram photo, it will similarly do well as a YouTube video thumbnail.

Let’s look at an example of how this works.

 

Current thumbnail. Image Credits: YouTube (opens in a new window)

 

We had the intrinsic popularity model look at three frames per second of this 23-minute video. It took about 20 minutes. The following were my favorites from the 20 images that had the highest overall scores.

Read Full Article

Bigger Problem

Bigger Problem

Ocean Solutions Accelerator’s third wave tackles a new set of aquatic challenges

The Sustainable Ocean Alliance and its Ocean Solutions Accelerator take on the problems facing our planet’s waters, and the latest cohort of companies in the latter show a fresh slate of issues to address and resources to utilize. From reef rehabilitation to a “Fitbit for fishing boats,” they’re trying to fix things up in the oceans or at least mitigate the damage we’re doing down there.

The accelerator’s four week, all-virtual (like all of them these days) program focuses on the unique challenges faced by social good companies in this space.

“Startups in the sector are still struggling to find adequate funding during the early phases of operations,” the accelerator’s co-founder Craig Dudenhoffer told TechCrunch in an email. “Many of the solutions (especially hardware) are costly to produce and take a heavy upfront cash investment. We found that out of the hundreds of applicants, only a fraction had received substantial investments. We believe more investors need to educate themselves on opportunities in the ocean sector.”

The SOA team selected nine companies for this wave, only three of which are U.S.-based. “This year, in spite of the COVID-19 pandemic, we saw our largest and most diverse applicant pool to date,” said Dudenhoffer in the release announcing the companies. “I was particularly encouraged by this year’s applicant pool to see the varying types of solutions, as well as an increase in the number of entrepreneurs that are actively building technologies to address the critical challenges that face the ocean.”

In the Accelerator over the Sea

SOA founder Daniela Fernandez recently noted that their area of operation is especially international, so keeping things virtual actually opens up a lot of possibilities, especially for smaller companies that can’t afford to temporarily relocate. “It gives you so many options and makes it far more inclusive,” she told me. “Everybody just has more flexibility and tranquility. So I believe we were headed in that direction anyway.”

Image Credits: ARC Marine

Here are the nine lucky companies:

• AquaAI (Norway): Developed a fishlike autonomous underwater vehicle for unobtrusive observation and inspection.
• AKUA (U.S.): Makes super-healthy kelp-based foods, starting with jerky and soon burgers.
• ARC Marine (U.K.): Helps protect and rehabilitate reefs with sustainable “Reef Cube” habitat and nursery.
• Desolenator (The Netherlands): Solar-powered desalination for communities facing fresh water shortages.
• FlyWire (U.S.): Digital catch monitoring for compliance with regulations and connected commerce.
• microTERRA (Mexico): Sustainable, aquafarm-grown protein for animal feed.
• Oceanworks (U.S.): Marketplace for recycled ocean-sourced plastic.
• PlanetCare (Slovenia): Filter for catching microfibers in washing machine drains before they enter the water system.
• Trademodo (Canada): New, comprehensive platform for ethical seafood businesses and supply chains.

The companies will get the tender loving care lavished on all the new accelerator’s participants, but possibly also a bit of harsh reality as they learn the difficulties of being an ethics-focused company with long-term goals in a capitalist system that demands almost immediate returns. One of the most important steps in building one of these companies seems to be getting over this demoralizing hump and seeing the possibilities in spite of the difficulties.

A demo day is scheduled for November 5, which is good timing because probably nothing else will be happening around then.

Read Full Article

Accessibility’s nextgen breakthroughs will be literally in your head

Predicting the future of technology for people with visual impairments is easier than you might think. In 2003, I wrote an article entitled “In the Palm of Your Hand” for the Journal of Visual Impairment & Blindness from the American Foundation for the Blind. The arrival of the iPhone was still four years away, but I was able to confidently predict the center of assistive technology shifting from the desktop PC to the smart phone. 

“A cell phone costing less than $100,” I wrote, “will be able to see for the person who can’t see, read for the person who can’t read, speak for the person who can’t speak, remember for the person who can’t remember, and guide the person who is lost.” Looking at the tech trends at the time, that transition was as inevitable as it might have seemed far-fetched.

We are at a similar point now, which is why I am excited to play a part of Sight Tech Global, a virtual event Dec. 2-3 that is convening the top technologists to discuss how AI and related technologies will usher in a new era of remarkable advances for accessibility and assistive tech, in particular for people who are blind or visually impaired.

To get to the future, let me turn to the past. I was walking around the German city of Speyer in the 1990s with pioneering blind assistive tech entrepreneur Joachim Frank. Joachim took me on a flight of fancy about what he really wanted from assistive technology, as opposed to what was then possible. He quickly highlighted three stories of how advanced tech could help him as he was walking down the street with me. 

• As I walk down the street, and walk by a supermarket, I do not want it to read all of the signs in the window. However, if one of the signs notes that kasseler kipchen (smoked porkchops, his favorite) are on sale, and the price is particularly good, I would like that whispered in my ear.
• And then, as a young woman approaches me walking in the opposite direction, I’d like to know if she’s wearing a wedding ring.
• Finally, I would like to know that someone has been following me for the last two blocks, that he is a known mugger, and that if I quicken my walking speed, go fifty meters ahead, turn right, and go another seventy meters, I will arrive at a police substation! 

Joachim blew my mind. In one short walk, he outlined a far bolder vision of what tech could do for him, without bogging down in the details. He wanted help with saving money, meeting new friends and keeping himself safe. He wanted abilities which not only equaled what people with normal vision had, but exceeded them. Above all, he wanted tools which knew him and his desires and needs. 

We are nearing the point where we can build Joachim’s dreams.  It won’t matter if the assistant whispers in your ear, or uses a direct neural implant to communicate. We will probably see both. But, the nexus of tech will move inside your head, and become a powerful instrument for equality of access. A new tech stack with perception as a service. Counter-measures to outsmart algorithmic discrimination. Tech personalization. Affordability. 

That experience will be built on an ever more application rich and readily available technology stack in the cloud. As all that gets cheaper and cheaper to access, product designers can create and experiment faster than ever. At first, it will be expensive, but not for long as adoption – probably by far more than simply disabled people – drives down price. I started my career in tech for the blind by introducing a reading machine that was a big deal because it halved the price of that technology to $5,000. Today even better OCR is a free app on any smartphone.

We could dive into more details of how we build Joachim’s dreams and meet the needs of millions of others of individuals with vision disabilities. But it will be far more interesting to explore with the world’s top experts at Sight Tech Global on Dec. 2-3 how those tech tools will become enabled In Your Head!

Registration is free and open to all. 

Jim Fruchterman a pioneering social good entrepreneur and founder of Benetech and Tech Matters.

Read Full Article

Hand-drawn illustrations of the world's weirdest plants | Nirupa Rao

Hand-drawn illustrations of the world's weirdest plants | Nirupa Rao

Botanical artist Nirupa Rao captures the spirit and beauty of nature in watercolor. With a portfolio of enchanting, scientifically accurate illustrations, she aims to reignite our emotional connection to the environment -- and open our eyes to an entire kingdom hidden in plain sight.

https://ift.tt/3nliU47

Click this link to view the TED Talk

Camera that will film a spacewalk in VR delivered to the International Space Station

One of the payloads aboard the International Space Station resupply mission that launched last Friday will providing a new perspective on one of the most enervating human experiences – the spacewalk. It’s a custom-made, 3D camera designed to capture content in 360-degrees while in space, and it will be used to film a spacewalk in immersive, cinematic VR for the first time ever on an upcoming ISS astronaut mission.

The camera is the result of a collaboration between Felix & Paul Studios, Time Studios, and in-space technology expert Nanoracks. It will ultimately be used to capture the footage that will then be used to produce a culminating episode of a series called Space Explorers: The ISS Series. To do that, it’ll be mounted on Nanoracks’ Kaber MicroSatellite deployer device, which will provide it with power, and allow it to be controlled via the Canadarm2 robotic arm that the ISS uses for manipulating external cargo. The team behind this says that the Canadian-made robot arm will essentially be used like a crane on a film set to capture the spacewalk of two actual ISS astronauts.

In terms of specs, the VR camera includes nine different 4K sensors, which can then stitch together a fully immersive 360-degree final image that’s rendered at 8K resolution. The camera, a Z-Cam V1 Pro, has been modified by Nanoracks using their expertise in creating equipment that can operate in and withstand the harsh environment of space – meaning it isn’t all that bothered by vacuum, UV radiation, ionizing radiation, plasma, wildly varying extreme temps that can go from -250 degrees Fahrenheit to +250 depending on sun exposure, and more. The enclosure for the camera is hermitically sealed, includes an aluminum radiation shield, and has both an active heating and passive cooling system, rendering it capable of surviving exposure to space for a full week.

The spacewalk will ultimately be aired via the Oculus Store, and you can already see the first two Space Explorers episode there right now if you have a compatible VR headset.

Read Full Article

Instagram expands shopping on IGTV, plans test of shopping on Reels

Instagram this morning announced the global expansion of its Instagram Shopping service across IGTV. The product, which lets you watch a video then checkout with a few taps, offers creators and influencers a way to more directly monetize their user base on Instagram, while also giving brands a way to sell merchandise to their followers. Instagram said it would also soon begin testing shopping within its newer feature and TikTok rival, Reels.

Image Credits: Instagram

Shopping has become a larger part of the Instagram experience over the past few years.

Instagram’s Explore section in 2018 gained a personalized Shopping channel filled with the things Instagram believed you’d want the most. It also expanded Shopping tags to Stories. Last year, it launched Checkout, a way to transact within the app when you saw something you wanted to buy. And just this summer, Instagram redesigned its dedicated Shop section, now powered by Facebook Pay.

Today, Instagram users can view products and make purchases across IGTV, Instagram Live, and Stories.

On IGTV, users can either complete the purchase via the in-app checkout or they can visit the seller’s website to buy. However, the expectation is that many shoppers will choose to pay for their items without leaving the app, for convenience’s sake. This allows Instagram to collect selling fees on those purchases. At scale, this can produce a new revenue stream for the company — particularly now as consumers shop online more than ever, due to the coronavirus pandemic’s acceleration of e-commerce.

In the future, Instagram says its shoppable IGTV videos will be made discoverable on Instagram Shop, as well.

Given its intention to make shopping a core part of the Instagram platform, it’s not surprising that the company intends to make Reels shoppable, too.

“Digital creators and brands help bring emerging culture to Instagram, and people come to Instagram to get inspired by them. By bringing shopping to IGTV and Reels, we’re making it easy to shop directly from videos. And in turn, helping sellers share their story, reach customers, and make a living,” said Instagram COO Justin Osofsky, in a statement.

Instagram isn’t alone in seeing the potential for shopping inspired by short-form video content. Walmart’s decision to try to acquire a stake in TikTok is tied to the growing “social commerce” trend which mixes together social media and online shopping to create a flurry of demand for new products — like a modern-day QVC aimed at Gen Z and broadcast across smartphones’ small screens.

By comparison, TikTok so far has only dabbled with social commerce. It has run select ad tests, like a partnership with Levi’s during the early days of the pandemic to create influencer-created ads that appeared in users’ feeds and directed users’ to Levi’s website. It has also experimented with allowing users to add links to e-commerce sites to TikTok profiles and other features.

Instagram didn’t say when Reels would gain shopping features, beyond “later this year.”

 

Read Full Article

Corsair’s TBT100 Thunderbolt 3 dock offers the right expandability in a well-designed package

Gaming peripheral expert Corsair has released a new Thunderbolt 3 docking station that boasts a number of useful ports, paired with aesthetics that should fit in perfectly with any of Apple’s Space Gray hardware kit. The TBT100 dock offers plenty of expandability for making your Mac the center of a temporary work-from-home office, or can provide great convenience and connection options even for more powerful desktop computer setups.

The basics

The Corsair TBT100 offers a full complement of ports powered via a single Thunderbolt 3 cable from your computer, along with a dedicated power adapter. For display, there are 2 HDMI 2.0 ports capable of 4K 60Hz output, with HDR color rendering. There are two USB 3.2 Type-C ports, one in front and one in back, as well as two USB 3.1 Type-A ports (both in back) that can all connect to both charge devices and provide data connections. A Gigabit Ethernet port provides networking, while a 3.5mm jack offers both headphone out and microphone in. There’s also an SDXC card reader that supports UHS-II speeds.

The TBT100 offers 85W power delivery via its lone Thunderbolt 3 cable for connected host notebooks, and can smart charge devices at up to 15W via the USB-C ports, or up to 7.5W via the USB-A connections.

Design and features

This is definitely one of the better-looking Thunderbolt 3 docks out there. It’s a category where it’s hard for design to stand out, since these generally all look roughly the same – metallic and plastic rectangles with a combinations of ports located front and back. Corsair’s dock doesn’t venture too far from this standard look, but the touches it adds like the gray aluminum finish and the way the aluminum continues around the rounded corners makes it a more attractive desktop addition than most.

The port arrangement is also well-conceived. Up front, there’s one USB-C port (handy for quickly plugging in a mobile device for a charge), the SD card reader (really useful for frequent use) and the 3.5 mm jack (ditto for commonly relocated items like headsets). Everything else is around back, letting you put more regularly connected cables in prime location for routing them to make them a more invisible part of your desktop setup.

Corsair’s choice to go with HDMI ports is also probably the best option on balance for most users. Many alternatives have gone with DisplayPort, but your average consumer these days is much more likely to have HDMI cables and HDMI-capable displays, and the spec still supports 4K resolution as well as HDR to get the most image quality out of any modern connected TV or monitor.

Bottom line

There are many flavors of Thunderbolt 3 docks, but the Corsair TBT100 offers a pretty perfect blend of connectivity, design and convenience relative to the pack. At $259.99, the price of the dock is also not too expensive, though it’s not cheap either. But if you’re looking for a reliable, permanent solution to a lack of connections for your home setup, this is the one to get.

Read Full Article

Nest Audio review

The Nest Audio is a surprisingly hefty thing. It’s compact, but dense, packing a lot into a footprint not much taller than an iPhone. At 2.65 pounds, it’s 2.5x the weight of the original Home. It’s clear that, above all, Google was interested in offering up something premium, in spite of a quite reasonable $99 price point.

It certainly took the company long enough. It’s been four years since the first device arrived — that’s a lifetime on the scale of smart home devices. But while Google introduced a slate of new products and delivered a key update to its smaller sibling, the Mini, its flagship smart speaker remained untouched, in all of its air freshener designed glory.

Google unveils the $99 Nest Audio smart speaker

In four years, Google has proven less interested in hardware upgrades than Amazon. That’s not a huge surprise from a company that’s long suggested that software — not hardware — is at the heart of product evolution. But even Google knows that software can only take you so far when it comes to things like premium audio. That’s where the new device comes in.

Image Credits: Brian Heater

Nest Audio’s name represents two key things. First, it’s the last of Google’s smart speaker/smart screen line to adopt the Nest title, cementing its 2014 Nest acquisition as its smart home brand. It’s not quite accurate to say that the Nest brand encompasses all of Google’s Home products — after all, a new Chromecast was launched this week with Google branding, but then, no one’s ever accused Google of being consistent about such things.

The Audio bit finds Google following a similar journey as Amazon. The first generation of smart speakers focused significantly more on the smart than the speaker. The devices were primarily considered a way to deliver smart assistants into the home — certainly not something that was set to replace anyone’s home stereo.

But Apple, for all of its issues bringing Siri into a home setting, proved that users were willing to invest in a premium product — so long as a company could demonstrate superior audio. Google followed up with the Home Max and Amazon did so first by beefing up the sound of the standard Echo, and more recently introducing the Echo Studio. There’s also the matter of the Samsung Galaxy Home, but the less said about the unreleased speaker — and Bixby in general — the better.

With Nest Audio, Google is looking to prove that good sound shouldn’t be the exclusive realm of high-end speakers. It even went so far as dropping $30 off the price of the original Google Home — putting it in line with current Echo pricing. The internals have been considerably upgraded, as well. A 50mm full-range driver (40mm on the Nest Mini) has been upgraded to a 75mm woofer for much stronger bass. Two passive radiators, meanwhile, have been swapped out for a 19mm tweeter to complete the picture.

Image Credits: Brian Heater

The speaker is also capable of getting really loud — 75% louder than the original Home, according to the company. It’s too loud for my apartment. Though I would advise against relying on a single speaker to fill a large space, as stuff gets distorted at peak volumes. A speaker of this caliber is best paired with another — which is, thankfully, something Google does reasonably well.

As it stands, the Nest Audio is quite clear and full, given its pricing and size. For space like the living room in my one-bedroom New York apartment, it’s got pretty good sound. The design means that (like the new Echo) you get pretty good audio from all sides — though the company cautions against, stay, sticking it deep on a bookshelf, or else you may deal with some muddy reverb. It’s clear that Google knew it had to step up its game to deal with superior audio from third-party speaker makers like Sony who have embraced Google Assistant, and it’s done a pretty good job here.

I still prefer the much heftier and massive Google Home Max that’s currently sitting by my computer on my desk. Size really does matter in the world of speakers, for a number of pragmatic reasons, including how it moves air to create sound. That said, you can currently purchase three Nest Audios for the price of a single Google Max, so that may be something worth considering, depending on your setup and the layout of your home.

Groups and pairing are one of the strongest reasons to consider these device. The Google Home app setup is extremely simple in that regard, and presents an extremely simple and fairly inexpensive way to set up a home audio system. You can either pair two of the same speakers to create stereo (a solid choice for, say, flanking the computer screen) or simply creating groups for multiple speakers to fill a space. I do the latter with my own home setup.

It’s usually a good solution, though even at this point in the life of the devices, it can still be pretty buggy. A lot of this comes down to Wi-Fi and connectivity issues, but it can be frustrating. Wireless systems are a lot easier — but less reliable — than simply wiring up your system the old-fashioned way. And of course, there’s the fact that the more wireless devices you install, the more strain there’s going to be on your home network.

Image Credits: Brian Heater

There are some nice tweaks to the system, as well. Ambient IQ actually turns up assistant’s voice when there are sounds in the background, while Media EQ dynamically adjusts the balance depending on what you’re listening to — be it music or a podcast, turning up the vocal output for the latter.

The speaker’s design has improved dramatically. We mocked the original Home for looking like a Glade air freshener since day one, and that criticism still stands. The Nest Audio, meanwhile, if far more unassuming. Covered entirely in fabric with a design that Google freely admits was inspired by a pillow, it’s designed to blend in with its décor — which, frankly, is precisely what a smart speaker should do.

There are five colors: white (chalk), black and Sage, Sand and Sky — all pastels. Odds are pretty good you’ll find one that fits your home. Google sent me a black one, which is likely what I would have chosen myself. And bonus points for the fact that the fabric is made from recycled water bottles, like the Nest Mini before it.

The Nest Audio is a long overdue upgrade to the company’s line of smart home devices and one that puts the focus on sound, precisely where it should be.

Read Full Article

Google delays mandating Play Store’s 30% cut in India to April 2022

Google is postponing the enforcement of its new Play Store billing policy in India to April 2022, days after more than 150 startups in the world’s second largest internet market forged an informal coalition to express concerns over the 30% charge the Android-maker plans to mandate on its store and started to explore an alternative marketplace for their apps.

The company, which is going live globally with the new Play Store rule in September 2021, is deferring the enforcement of the policy only in India, it said. It is also listening to developers and willing to engage to allay their concerns, it said.

Last week, Google said it would no longer allow any apps to circumvent its payment system within the Play Store. The move, pitched by Google as a “clarification” of its existing policy, would allow the company to ensure it gets as high as a 30% cut on in-app purchases made through Android apps operating in a range of a categories.

Google’s announcement today is a direct response to the loudest scrutiny it has received in a decade in India — its biggest market by users but also a place where, compared to Western markets, it generates little revenue. More than 150 startups in India last week formed an informal coalition to fight the company’s strong hold on Indian app ecosystem. Google commands 99% of the smartphone market in India, according to research firm Counterpoint.

Among the startups that have expressed concerns over Google’s new policy are Paytm, India’s most valuable startup, payments processor Razorpay, fantasy sports firm Dream11, social network ShareChat, and business e-commerce IndiaMART.

More than 50 Indian executives relayed these concerns to India’s Ministry of Electronics and Information Technology over a video call on Saturday, according to three people who attended the call.

Several businesses in India have long expressed concerns with the way Google has enforced its policies in India, but the matter escalated last month after the company temporarily pulled Paytm app from the Play Store for promoting gambling.

Google said Paytm had repeatedly violated its policies, and the company’s Play Store has long prohibited apps that promote gambling in India. Google has sent notices about warnings over gambling to several more firms in India in recent weeks.

A senior industry executive told TechCrunch that the company should have expressed these concerns months before the popular cricket tournament IPL was scheduled to commence. Fantasy sports apps allow users to pick their favorite players and teams. These players stand to win real money or points that they can redeem for physical goods purchase based on the real-world performance of their preferred teams and players. IPL season sees a huge surge in popularity of such fantasy sports apps.

“The IPL even got delayed by months. Why did Google wait for so long? And why does the company have a problem with so-called gambling in India, when it permits such activities in other markets? The Indian government has no problem with it,” the executive said, requesting anonymity.

Paytm on Monday announced its own mini-app store featuring several popular services including ride-hailing firm Ola, health care provides 1mg and Practo, fitness startup Cure.fit, music-streaming service Gaana, car-rental provider Zoomcar, Booking.com, and eateries Faasos, Domino’s Pizza, and McDonald’s. The startup claimed that more than 300 firms have signed up for its mini store and that its app reaches more than 150 million users each month. (In a written statement to TechCrunch, Paytm said in June its app reached more than 50 million users in India each month.

Paytm, which says its mini-app store is open to any developer, will provide a range of features including the ability to support subscriptions and one-step login. The startup, which claims  said it will not charge any commission to developers for using its payments system or UPI payments infrastructure, but will levy a 2% charge on “other instruments such as credit cards.”

“There are many challenges with traditional mobile apps such as maintaining multiple codebases across platforms (iOS, Android or Web), costly user acquisition and requirement of app release and then a waiting period for user adoption for any change made in the app. Launching as a Mini Apps gives you freedom from all these hassles: implying lesser development/testing and maintenance costs which help you reach millions of Paytm users in a Jiffy,” the Indian firm said in its pitch.

The launch of a mini-store further cements Alibaba-backed Paytm’s push into turning itself into a super-app. Its chief rivals, Walmart-backed PhonePe and Google Pay, also operate similar mini stores on their apps.

Whether Paytm’s own mini app store and postponement of Google’s new Play Store policy are enough to calm other startups’ complaints remain to be seen. PhonePe is not one of the mini apps on Paytm’s store, a Paytm spokesperson told TechCrunch.

“I am proud that we are today launching something that creates an opportunity for every Indian app developer. Paytm mini app store empowers our young Indian developers to leverage our reach and payments to build new innovative services,” said Vijay Shekhar Sharma, co-founder and chief executive of Paytm, in a statement.

Read Full Article

Google research lets sign language switch ‘active speaker’ in video calls

An aspect of video calls that many of us take for granted is the way they can switch between feeds to highlight whoever’s speaking. Great — if speaking is how you communicate. Silent speech like sign language doesn’t trigger those algorithms, unfortunately, but this research from Google might change that.

It’s a real-time sign language detection engine that can tell when someone is signing (as opposed to just moving around) and when they’re done. Of course it’s trivial for humans to tell this sort of thing, but it’s harder for a video call system that’s used to just pushing pixels.

A new paper from Google researchers, presented (virtually, of course) at ECCV, shows how it can be done efficiency and with very little latency. It would defeat the point if the sign language detection worked but it resulted in delayed or degraded video, so their goal was to make sure the model was both lightweight and reliable.

This hand-tracking algorithm could lead to sign language recognition

The system first runs the video through a model called PoseNet, which estimates the positions of the body and limbs in each frame. This simplified visual information (essentially a stick figure) is sent to a model trained on pose data from video of people using German Sign Language, and it compares the live image to what it thinks signing looks like.

Image Credits: Google

This simple process already produces 80 percent accuracy in predicting whether a person is signing or not, and with some additional optimizing gets up to 91.5 percent accuracy. Considering how the “active speaker” detection on most calls is only so-so at telling whether a person is talking or coughing, those numbers are pretty respectable.

In order to work without adding some new “a person is signing” signal to existing calls, the system pulls clever a little trick. It uses a virtual audio source to generate a 20 kHz tone, which is outside the range of human hearing, but noticed by computer audio systems. This signal is generated whenever the person is signing, making the speech detection algorithms think that they are speaking out loud.

Right now it’s just a demo, which you can try here, but there doesn’t seem to be any reason why it couldn’t be built right into existing video call systems or even as an app that piggybacks on them. You can read the full paper here.

iOS 14 lets deaf users set alerts for important sounds, among other clever accessibility perks

Read Full Article

Twitter will make users remove tweets hoping Trump dies of COVID-19

President Donald Trump’s positive COVID-19 result has made Twitter a busy place in the past 24 hours, including some tweets that have publicly wished — some subtly and others more directly — that he die from the disease caused by coronavirus.

Twitter put out a reminder to folks that it doesn’t allow tweets that wish or hope for death or serious bodily harm or fatal disease against anyone. Tweets that violate this policy will need to be removed, Twitter said Friday. However, it also clarified that this does not automatically mean suspension. Several new outlets misreported that users would be suspended automatically. Of course, that doesn’t mean users won’t be suspended.

tweets that wish or hope for death, serious bodily harm or fatal disease against *anyone* are not allowed and will need to be removed. this does not automatically mean suspension. https://t.co/lQ8wWGL2y0 https://t.co/P2vGfUeUQf

— Twitter Comms (@TwitterComms) October 2, 2020

On Thursday evening, Trump tweeted that he and his wife and First Lady Melania Trump had tested positive for COVID-19. White House physician Sean Conley issued a memo Friday confirming the positive results of SAR-Cov-2 virus, which often more commonly known as COVID-19.  Trump was seen boarding a helicopter Friday evening that was bound for Walter Reed Medical Center for several days of treatment.

The diagnosis sent shares tumbling Friday on the key exchanges, including Nasdaq. The news put downward pressure on all major American indices, but heaviest on tech shares.

Read Full Article

Daily Crunch: Twitter confronts image-cropping concerns

Twitter addresses questions of bias in its image-cropping algorithms, we take a look at Mario Kart Live and the stock market takes a hit after President Trump’s COVID-19 diagnosis. This is your Daily Crunch for October 2, 2020.

The big story: Twitter confronts image-cropping concerns

Last month, (white) PhD student Colin Madland highlighted potential algorithmic bias on Twitter and Zoom — in Twitter’s case, because its automatic image cropping seemed to consistently highlight Madland’s face over that of a Black colleague.

Today, Twitter said it has been looking into the issue: “While our analyses to date haven’t shown racial or gender bias, we recognize that the way we automatically crop photos means there is a potential for harm.”

Does that mean it will stop automatically cropping images? The company said it’s “exploring different options” and added, “We hope that giving people more choices for image cropping and previewing what they’ll look like in the tweet composer may help reduce the risk of harm.”

The tech giants

Nintendo’s new RC Mario Kart looks terrific — Mario Kart Live (with a real-world race car) makes for one hell of an impressive demo.

Tesla delivers 139,300 vehicles in Q3, beating expectations — Tesla’s numbers in the third quarter marked a 43% improvement from the same period last year.

Zynga completes its acquisition of hyper-casual game maker Rollic — CEO Frank Gibeau told me that this represents Zynga’s first move into the world of hyper-casual games.

Startups, funding and venture capital

Elon Musk says an update for SpaceX’s Starship spacecraft development program is coming in 3 weeks —  Starship is a next-generation, fully reusable spacecraft that the company is developing with the aim of replacing all of its launch vehicles.

Paired picks up $1M funding and launches its relationship app for couples — Paired combines audio tips from experts with “fun daily questions and quizzes” that partners answer together.

With $2.7M in fresh funding, Sora hopes to bring virtual high school to the mainstream — Long before the coronavirus, Sora was toying with the idea of live, virtual high school.

Advice and analysis from Extra Crunch

Spain’s startup ecosystem: 9 investors on remote work, green shoots and 2020 trends — While main hubs Madrid and Barcelona bump heads politically, tech ecosystems in each city have been developing with local support.

Which neobanks will rise or fall? — Neobanks have led the $3.6 billion in venture capital funding for consumer fintech startups this year.

Asana’s strong direct listing lights alternative path to public market for SaaS startups — Despite rising cash burn and losses, Wall Street welcomed the productivity company.

Everything else

American stocks drop in wake of president’s COVID-19 diagnosis — The news is weighing heavily on all major American indices, but heaviest on tech shares.

Digital vote-by-mail applications in most states are inaccessible to people with disabilities — According to an audit by Deque, most states don’t actually have an accessible digital application.

The Daily Crunch is TechCrunch’s roundup of our biggest and most important stories. If you’d like to get this delivered to your inbox every day at around 3pm Pacific, you can subscribe here.

Read Full Article

Twitter is building ‘Birdwatch,’ a system to fight misinformation by adding more context to tweets

Twitter is developing a new product called “Birdwatch,” which the company confirms is an attempt at addressing misinformation across its platform by providing more context for tweets, in the form of notes. Tweets can be added to “Birdwatch” — meaning flagged for moderation — from the tweet’s drop-down menu, where other blocking and reporting tools are found today. A small binoculars icon will also appear on tweets published to the Twitter Timeline. When the button is clicked, users are directed to a screen where they can view the tweet’s history of notes.

Based on screenshots of Birdwatch unearthed through reverse engineering techniques, a new tab called “Birdwatch Notes” will be added to Twitter’s sidebar navigation, alongside other existing features like Lists, Topics, Bookmarks and Moments.

This section will allow you to keep track of your own contributions, aka your “Birdwatch Notes.”

To detect fake news, this AI first learned to write it

The feature was first uncovered this summer in early stages of development by reverse engineer Jane Manchun Wong, who found the system through Twitter’s website. At the time, Birdwatch didn’t have a name, but it clearly showed an interface for flagging tweets, voting on whether or not the tweet was misleading, and adding a note with further explanations.

Twitter is working on a moderation tool to monitor misinformations on Twitter

Moderators can flag tweets, vote on whether it is misleading, and add a note about it

(I made up my own note to show what it currently looks like) pic.twitter.com/YIa6zt58Fj

— Jane Manchun Wong (@wongmjane) August 5, 2020

Twitter updated its web app a few days after her discovery, limiting further investigation.

This week, however, a very similar interface was again discovered in Twitter’s code, this time on iOS.

Twitter adds a warning label fact-checking Trump’s false voting claims

According to social media consultant Matt Navarra, who tweeted several more screenshots of the feature on mobile, Birdwatch allows users to attach notes to a tweet. These notes can be viewed when clicking on the binoculars button on the tweet itself.

In other words, additional context about the statements made in the tweet would be open to the public.

New Twitter feature?

‘Birdwatch’?!

Any guesses what it does? pic.twitter.com/1vmpZdTKLi

— Matt Navarra (@MattNavarra) September 30, 2020

What’s less clear is whether everyone on Twitter will be given access to annotate tweets with additional context, or whether this permission will require approval, or only be open to select users or fact checkers.

Twitter early adopter and hashtag inventor Chris Messina openly wondered if Birdwatch could be some sort of “citizen’s watch” system for policing disinformation on Twitter. It turns out, he was right.

Wondering out loud if Twitter's #Birdwatch feature might turn into a kind of "citizen's watch" for disinformation on Twitter… https://t.co/N1YlVkbu8m

— Chris Messina (@chrismessina) October 1, 2020

According to line items he found within Twitter’s code, these annotations — the “Birdwatch Notes” — are referred to as “contributions,” which does seem to imply a crowdsourced system. (After all, a user would contribute to a shared system, not to a note they were writing for only themselves to see.)

Image Credits: Chris Messina

Crowdsourcing moderation wouldn’t be new to Twitter. For several years, Twitter’s live-streaming app Periscope has relied on crowdsourcing techniques to moderate comments on its real-time streams in order to clamp down on abuse.

There is still much we don’t know about how Birdwatch will work from a non-technical perspective, however. We don’t know if everyone will have the same abilities to annotate tweets, how attempts to troll this system will be handled, or what would happen to a tweet if it got too many negative dings, for example.

In more recent months, Twitter has tried to take a harder stance on tweets that contain misleading, false or incendiary statements. It has even gone so far as to apply fact-check labels to some of Trump’s tweets and has hidden others behind a notice warning users that the tweet has violated Twitter’s rules. But scaling moderation across all of Twitter is a task the company has not been well-prepared for, as it built for scale first, then tried to figure out policies and procedures around harmful content after the fact.

Reached for comment, Twitter declined to offer details regarding its plans for Birdwatch, but did confirm the feature was designed to combat the spread of misinformation.

“We’re exploring a number of ways to address misinformation and provide more context for tweets on Twitter,” a Twitter spokesperson told TechCrunch. “Misinformation is a critical issue and we will be testing many different ways to address it,” they added.

WTF is fake news?

 

Read Full Article

Google wakes up from its VR daydream

Daydream, Google’s mobile-focused virtual reality platform is losing official support from Google, Android Police reports. The company confirmed that it will no longer be updating the Daydream software, with the publication noting that “Daydream may not even work on Android 11” as a result of this.

This isn’t surprising to anyone who has been tracking the company’s moves in the space. After aggressive product rollouts in 2016 and 2017, Google quickly abandoned its VR efforts which, much like the Samsung Gear VR, allowed users to drop a compatible phone into a headset holster and use the phone’s display and compute to power VR experiences. After Apple’s announcement of ARKit, the company did a hard pivot away from VR, turning its specialty AR platform Tango into ARCore, an AR developer platform that has also not seen very much attention from Google in recent months.

Google bowing out of official support from Daydream comes after years without product updates to their own View headset and very little investment in their content ecosystem which wrecked the chances of Lenovo’s third-party effort the standalone Mirage Solo.

What went wrong? Once it became clear that Daydream wasn’t going to be an easy win, they kind of just abandoned the effort. Google’s hardware business is already peanuts to their search and ads business so it probably wasn’t clear what the point was, but virtual reality also quickly went from being the “it” technology to work on to clearly being a labor of love for a select few. Google determined it wasn’t the effort while Facebook continued to double down. It’s hard to fault them for it, in 2020, even with some very good hardware on the way from Oculus, it still isn’t clear what VR’s future looks like.

It is clear, however, that Daydream won’t be part of it.

Review: Facebook’s Oculus Quest 2 is outstanding

Read Full Article

Macrometa, an edge computing service for app developers, lands $7M seed round led by DNX

As people continue to work and study from home because of the COVID-19 pandemic, interest in edge computing has increased. Macrometa, a Palo Alto-based startup that provides edge computing infrastructure for app developers, announced today it has closed a $7 million seed round.

The funding was led by DNX Ventures, an investment fund that focuses on early-stage B2B startups. Other participants included returning investors Benhamou Global Ventures, Partech Partners, Fusion Fund, Sway Ventures, Velar Capital and Shasta Ventures.

DNX Ventures launches $315 million fund for US and Japanese B2B startups

While cloud computing relies on servers and data centers owned by providers like Amazon, IBM, Microsoft and Google, edge computing is geographically distributed, with computing done closer to data sources, allowing for faster performance.

Founded in 2018 by chief executive Chetan Venkatesh and chief architect Durga Gokina, Macrometa’s globally distributed data service, called Global Data Network, combines a distributed NoSQL database and a low-latency stream data processing engine. It allows developers to run their cloud apps and APIs across 175 edge regions around the world. To reduce delays, app requests are sent to the region closest to the user. Macrometa claims that requests can be processed in less than 50 milliseconds globally, making it 50 to 100 times faster than cloud platforms like DyanmoDB, MongoDB or Firebase. One of the ways that Macrometa differentiates from competitors is that it enables developers to work with data stored across a global network of cloud providers, like Google Cloud and Amazon Web Services (for example), instead of a single provider.

As more telecoms roll out 5G networks, demand for globally distributed, serverless data computing services like Macrometa are expected to increase, especially to support enterprise software. Other edge computing-related startups that have recently raised funding include Latent AI, SiMa.ai and Pensando.

A spokesperson for Macrometa said the seed round was oversubscribed because the pandemic has increased investor interest in cloud and edge companies like Snowflake, which recently held its initial public offering.

JFrog and Snowflake’s aggressive IPO pricing point to strong demand for cloud shares

Macrometa also announced today that it has added to its board of directors DNX managing partner Q Motiwala, former Auth0 and xnor.ai chief executive Jon Gelsey and Armorblox chief technology officer Rob Fry.

In a statement about the funding, Motiwala said, “As we look at the next five to ten years of cloud evolution, it’s clear to us that enterprise developers need a platform like Macrometa to go beyond the constraints, scaling limitations and high-cost economics that current cloud architecture impose. What Macrometa is doing for edge computing, is what Amazon Web Services did for the cloud a decade ago.”

Edge computing could push the cloud to the fringe

Read Full Article

Massively Large-Scale Distributed Reinforcement Learning with Menger

Posted by Amir Yazdanbakhsh, Research Scientist, and Junchaeo Chen, Software Engineer, Google Research

In the last decade, reinforcement learning (RL) has become one of the most promising research areas in machine learning and has demonstrated great potential for solving sophisticated real-world problems, such as chip placement and resource management, and solving challenging games (e.g., Go, Dota 2, and hide-and-seek). In simplest terms, an RL infrastructure is a loop of data collection and training, where actors explore the environment and collect samples, which are then sent to the learners to train and update the model. Most current RL techniques require many iterations over batches of millions of samples from the environment to learn a target task (e.g., Dota 2 learns from batches of 2 million frames every 2 seconds). As such, an RL infrastructure should not only scale efficiently (e.g., increase the number of actors) and collect an immense number of samples, but also be able to swiftly iterate over these extensive amounts of samples during training.

Overview of an RL system in which an actor sends trajectories (e.g., multiple samples) to a learner. The learner trains a model using the sampled data and pushes the updated model back to the actor (e.g. TF-Agents, IMPALA).

Today we introduce Menger¹, a massive large-scale distributed RL infrastructure with localized inference that scales up to several thousand actors across multiple processing clusters (e.g., Borg cells), reducing the overall training time in the task of chip placement. In this post we describe how we implement Menger using Google TPU accelerators for fast training iterations, and present its performance and scalability on the challenging task of chip placement. Menger reduces the training time by up to 8.6x compared to a baseline implementation.

Menger System Design
There are various distributed RL systems, such as Acme and SEED RL, each of which focus on optimizing a single particular design point in the space of distributed reinforcement learning systems. For example, while Acme uses local inference on each actor with frequent model retrieval from the learner, SEED RL benefits from a centralized inference design by allocating a portion of TPU cores for performing batched calls. The tradeoffs between these design points are (1) paying the communication cost of sending/receiving observations and actions to/from a centralized inference server or paying the communication cost of model retrieval from a learner and (2) the cost of inference on actors (e.g., CPUs) compared to accelerators (e.g., TPUs/GPUs). Because of the requirements of our target application (e.g., size of observations, actions, and model size), Menger uses local inference in a manner similar to Acme, but pushes the scalability of actors to virtually an unbounded limit. The main challenges to achieving massive scalability and fast training on accelerators include:

1. Servicing a large number of read requests from actors to a learner for model retrieval can easily throttle the learner and quickly become a major bottleneck (e.g., significantly increasing the convergence time) as the number of actors increases.
2. The TPU performance is often limited by the efficiency of the input pipeline in feeding the training data to the TPU compute cores. As the number of TPU compute cores increases (e.g., TPU Pod), the performance of the input pipeline becomes even more critical for the overall training runtime.

Efficient Model Retrieval
To address the first challenge, we introduce transparent and distributed caching components between the learner and the actors optimized in TensorFlow and backed by Reverb (similar approach used in Dota). The main responsibility of the caching components is to strike a balance between the large number of requests from actors and the learner job. Adding these caching components not only significantly reduces the pressure on the learner to service the read requests, but also further distributes the actors across multiple Borg cells with a marginal communication overhead. In our study, we show that for a 16 MB model with 512 actors, the introduced caching components reduce the average read latency by a factor of ~4.0x leading to faster training iterations, especially for on-policy algorithms such as PPO.

Overview of a distributed RL system with multiple actors placed in different Borg cells. Servicing the frequent model update requests from a massive number of actors across different Borg cells throttles the learner and the communication network between learner and actors, which leads to a significant increase in the overall convergence time. The dashed lines represent gRPC communication between different machines.

Overview of a distributed RL system with multiple actors placed in different Borg cells with the introduced transparent and distributed caching service. The learner only sends the updated model to the distributed caching services. Each caching service handles the model request updates from the nearby actors (i.e., actors placed on the same Borg cells) and the caching service. The caching service not only reduces the load on the learner for servicing the model update requests, but also reduces the average read latency by the actors.

High Throughput Input Pipeline
To deliver a high throughput input data pipeline, Menger uses Reverb, a recently open-sourced data storage system designed for machine learning applications that provides an efficient and flexible platform to implement experience replay in a variety of on-policy/off-policy algorithms. However, using a single Reverb replay buffer service does not currently scale well in a distributed RL setting with thousands of actors, and simply becomes inefficient in terms of write throughput from actors.

A distributed RL system with a single replay buffer. Servicing a massive number of write requests from actors throttles the replay buffer and reduces its overall throughput. In addition, as we scale the learner to a setting with multiple compute engines (e.g., TPU Pod), feeding the data to these engines from a single replay buffer service becomes inefficient, which negatively impacts the overall convergence time.

To better understand the efficiency of the replay buffer in a distributed setting, we evaluate the average write latency for various payload sizes from 16 MB to 512 MB and a number of actors ranging from 16 to 2048. We repeat the experiment when the replay buffer and actors are placed on the same Borg cell. As the number of actors grows the average write latency also increases significantly. Expanding the number of actors from 16 to 2048, the average write latency increases by a factor of ~6.2x and ~18.9x for payload size 16 MB and 512 MB, respectively. This increase in the write latency negatively impacts the data collection time and leads to inefficiency in the overall training time.

The average write latency to a single Reverb replay buffer for various payload sizes (16 MB - 512 MB) and various number of actors (16 to 2048) when the actors and replay buffer are placed on the same Borg cells.

To mitigate this, we use the sharding capability provided by Reverb to increase the throughput between actors, learner, and replay buffer services. Sharding balances the write load from the large number of actors across multiple replay buffer servers, instead of throttling a single replay buffer server, and also minimizes the average write latency for each replay buffer server (as fewer actors share the same server). This enables Menger to scale efficiently to thousands of actors across multiple Borg cells.

A distributed RL system with sharded replay buffers. Each replay buffer service is a dedicated data storage for a collection of actors, generally located on the same Borg cells. In addition, the sharded replay buffer configuration provides a higher throughput input pipeline to the accelerator cores.

Case Study: Chip Placement
We studied the benefits of Menger in the complex task of chip placement for a large netlist. Using 512 TPU cores, Menger achieves significant improvements in the training time (up to ~8.6x, reducing the training time from ~8.6 hours down to merely one hour in the fastest configuration) compared to a strong baseline. While Menger was optimized for TPUs, that the key factor for this performance gain is the architecture, and we would expect to see similar gains when tailored to use on GPUs.

The improvement in training time using Menger with variable number of TPU cores compared to a baseline in the task of chip placement.

We believe that Menger infrastructure and its promising results in the intricate task of chip placement demonstrate an innovative path forward to further shorten the chip design cycle and has the potential to not only enable further innovations in the chip design process, but other challenging real-world tasks as well.

Acknowledgments
Most of the work was done by Amir Yazdanbakhsh, Junchaeo Chen, and Yu Zheng. We would like to also thank Robert Ormandi, Ebrahim Songhori, Shen Wang, TF-Agents team, Albin Cassirer, Aviral Kumar, James Laudon, John Wilkes, Joe Jiang, Milad Hashemi, Sat Chatterjee, Piotr Stanczyk, Sabela Ramos, Lasse Espeholt, Marcin Michalski, Sam Fishman, Ruoxin Sang, Azalia Mirhosseini, Anna Goldie, and Eric Johnson for their help and support.

---

1 A Menger cube is a three-dimensional fractal curve, and the inspiration for the name of this system, given that the proposed infrastructure can virtually scale ad infinitum.^↩