Replace non-stop Zoom with remote office avatars app Pragli

Could avatars that show what co-workers are up to save work-from-home teams from constant distraction and loneliness? That’s the idea behind Pragli, the Bitmoji for the enterprise. It’s a virtual office app that makes you actually feel like you’re in the same building.

Pragli uses avatars to signal whether co-workers are at their desk, away, in a meeting, in the zone while listening to Spotify, taking a break at a digital virtual watercoooler, or done for the day. From there, you’ll know whether to do a quick ad-hoc audio call, cooperate via screenshare, schedule a deeper video meeting, or a send a chat message they can respond to later. Essentially, it translates the real word presence cues we use to coordinate collaboration into an online workplace for distributed teams.

“What Slack did for email, we want to do for video conferencing” Pragli co-founder Doug Safreno tells me. “Traditional video conferencing is exclusive by design, whereas Pragli is inclusive. Just like in an office, you can see who is talking to who.” That means less time wasted planning meetings, interrupting colleagues who are in flow, or waiting for critical responses. Pragli offers the focus that makes remote work productive with the togetherness that keeps everyone sane and in sync.

The idea is to solve the top three problems that Pragli’s extensive interviews and a Buffer/AngelList study discovered workers hate:

1. Communication friction
2. Loneliness
3. Lack of boundaries boundaries

You never have to worry about whether you’re intruding on someone’s meeting, or if it’d be quicker to hash something out on a call instead of vague text. Avatars give remote workers a sense of identity, while the Pragli watercooler provides a temporary place to socialize rather than an endless Slack flood of GIFs. And since you clock in and out of the Pragli office just like a real one, co-workers understand when you’ll reply quickly versus when you’ll respond tomorrow unless there’s an emergency.

“In Pragli, you log into the office in the morning and there’s a clear sense of when I’m working and when I’m not working. Slack doesn’t give you a strong sense if they’re online or offline” Safreno explains. “Everyone stays online and feels pressured to respond at any time of day.”

Pragli co-founder Doug Safreno

Safreno and his co-founder Vivek Nair know the feeling first-hand. After both graduating in computer science from Stanford, they built StacksWare to help enterprise software customers avoid overpaying by accurately measuring their usage. But when they sold StacksWare to Avi Networks, they spent two years working remotely for the acquirer. The friction and loneliness quickly crept in.

They’d message someone, not hear back for a while, then go back and forth trying to discuss the problem before eventually scheduling a call. Jumping into synchronous communicating would have been much more efficient. “The loneliness was more subtle, but it built up after the first few weeks” Safreno recalls. “We simply didn’t socially bond while working remotely as well as in the office. Being lonely was de-motivating, and it negatively affected our productivity.”

The founders interviewed 100 remote engineers, and discovered that outside of scheduled meetings, they only had one audio or video call with co-workers per week. That convinced them to start Pragli a year ago to give work-from-home teams a visual, virtual facsimile of a real office. With no other full-time employees, the founders built and released a beta of Pragli last year. Usage grew 6X in March and is up 20X since January 1st.

Today Pragli officially launches, and it’s free until June 1st. Then it plans to become freemium with the full experience reserved for companies that pay per user per month. Pragli is also announcing a small pre-seed round today led by K9 Ventures, inspired by the firm’s delight using the product itself.

To get started with Pragi, teammates download the Pragli desktop app and sign in with Google, Microsoft, or Github. User then customize their avatar with a wide range of face, hair, skin, and clothing options. It can use your mouse and keyboard interaction to show if you’re at your desk or not, or use your webcam to translate occasional snapshots of your facial expressions to your avatar. You can also connect your Spotify and calendar to show you’re listening to music (and might be concentrating), reveal or hide details of your meeting, and decide whether people can ask to interrupt your or that you’re totally unavailable.

From there, you can communicate by audio, video, or text with any of your available co-workers. Guests can join conversations via the web and mobile too, though the team is working on a full-fledged app for phones and tablets. Tap on someone and you can instantly talk to them, though their mic stays muted until they respond. Alternatively, you can jump into Slack-esque channels for discussing specific topics or holding recurring meetings. And if you need some downtime, you can hang out in the watercooler or trivia game channel, or set a manual away message.

Pragli has put a remarkable amount of consideration into how the little office social cues about when to interrupt someone translate online, like if someone’s wearing headphones, in a deep convo already, or if they’re chilling in the microkitchen. It’s leagues better than having no idea what someone’s doing on the other side of Slack or what’s going on in a Zoom call. It’s a true virtual office without the clunky VR headset.

“Nothing we’ve tried has delivered the natural, water-cooler-style conversations that we get from Pragli” says Storj Labs VP of engineering JT Olio. “The ability to switch between ‘rooms’ with screen sharing, video, and voice in one app is great. It has really helped us improve transparency across teams. Plus, the avatars are quite charming as well.”

With Microsoft’s lack of social experience, Zoom consumed with its scaling challenges, and Slack doubling down on text as it prioritizes Zoom integration over its own visual communication features, there’s plenty of room for Pragli to flourish. Meanwhile, COVID-19 quarantines are turning the whole world towards remote work, and it’s likely to stick afterwards as companies deemphasize office space and hire more abroad.

The biggest challenge will be making such a broad product encompassing every communication medium and tons of new behaviors comprehensible enough to onboard whole teams. “How do you build a product that doesn’t feel distracting like Slack but where people can still have the spontaneous conversations that are so important to companies innovating?” Safreno asks. The Pragli founders are also debating how to encompass mobile without making people feel like the office stalks them after hours.

“Long-term, [Pragli] should be better than being in the office because you don’t actually have to walk around looking for [co-workers], and you get to decide how you’re presented” Safreno concludes. “We won’t quit because we want to work remotely for the rest of our lives.”

Read Full Article

The 8 Best Skateboarding Games of All Time

Video game genres peak in cycles. A few great skateboarding and snowboarding games will release at the same time, and then fans have no new titles for years. The time when a new skateboarding, motocross, or other action sports game would appear every year seems long gone.

Still, there are many skateboarding classics worth checking out once more. Better still, a few new skateboarding games are hitting the market, filling the void. With that in mind, here are the best skateboarding games ever made—including a new skateboarding game you should try.

1. Tony Hawk’s Pro Skater 3 (2001)

You could debate the merits of the various “Pro Skater” games all day. But many players agree that the third title in the series reigns supreme. Even though it was released over 15 years ago, it’s still one of the highest-ranked video games of all time on Metacritic. It also holds the honor of being the #1 rated game on the PlayStation 2. If you haven’t played it, you’re missing out.

It took the classic formula set out by Tony Hawk’s Pro Skater and Pro Skater 2, and built upon it. There weren’t any big changes from the previous titles, other than the addition of the revert, which let players string even more tricks together. But delightful new levels like the Foundry, Skater Island, and the Cruise Ship made their first appearance. And a host of new secret characters showed up, including Darth Maul, Doom Guy, and the Demoness.

The game’s signature arcade-like feeling is perfectly on point in this game. It’s no big deal to throw a 900-degree spin or grind for miles at a time with a few button presses. Pick a level, choose a skater, and get ready to shred. Ridiculous tricks like the Human Dart grind, the Pizza Guy, and the Sith Saber Spin cement the fantastic, absurd tone of the game. It’s an absolute blast.

If you don’t have a PlayStation 2 lying around, your best bet for playing THPS 3 is to use a PlayStation 2 emulator on your PC. Tony Hawk’s Pro Skater HD and the Revert Pack give you some of what made the game great on PS3 or Xbox 360, but it’s not the full game experience.

2. Skate (2007)

When Tony Hawk was ruling the streets, EA’s Skate came in with something new. Its control system did away with simple button presses and replaced it with a more “organic” interface. By moving the left and the right sticks in specific patterns, meant to mimic the movement of a skater’s feet, you make your skater do different tricks.

While the control system can be challenging to learn, it has a unique appeal. Unlike the arcade feel of THPS, Skate gives you a more authentic experience. Landing a nollie heelflip is more difficult than just pressing a few buttons. If you’re used to the Tony Hawk’s Pro Skater method of control, making the transition brings a bit of a shock. But it’s worth it when you pull off satisfying tricks.

The story-driven action mimics that of Tony Hawk’s Underground. You travel around the city, completing challenges and garnering acclaim. But the open-ended nature of the game means you can do whatever you want. Skate around San Vanelona to find the best spots. Learn a crazy new trick that you discovered. Or just ride around and chill. It’s totally up to you.

Like Pro Skater 3, if you want to play Skate, you’re going to need to either dig up an older system or emulate it, which is more difficult for PS3 and Xbox 360 games. The difficulty is emulating the specific hardware for each device and creating a stable environment for the game to run.

The emulators that do work—RPRCS3 for the PlayStation 3 and Xenia for the Xbox 360—cannot guarantee you game will work. Furthermore, emulating games at this level places the host system under a huge amount of strain because of all the components that must be reproduced.

3. Tony Hawk’s Underground (2003)

Skate may have redefined the genre, but Tony Hawk’s Underground set the bar high for story-based skateboarding games. Playing as yourself instead of a pro skater was a departure from previous Pro Skater games, as was the “punk-to-pro” storyline. The environments of Underground were likewise different from previous titles, with an emphasis on street skating over parks and crazy dream locations like Area 51.

The story is entertaining (if a bit far-fetched), the skating is as fun as ever, and many opportunities for customization make you feel like the protagonist skater is really your own. The ability to explore a variety of cities gives you plenty of great skating options, but the replay value of Underground does feel quite limited compared to other entries.

You can probably guess what I’m about to say next: you should emulate this one if you don’t own an older console.

4. Skate 2 (2009)

By sticking with the same formula as the first game but beefing up pretty much every area, Skate 2 surpassed its predecessor in the series. Still challenging and rewarding, the flick-it control system continued to differentiate this series from the Hawk games. With a significantly larger bag of tricks than the first, Skate 2 will keep you practicing and learning for hours.

The slightly dystopian storyline also stands out among skateboarding games. Players take on Mongocorp, the giant corporation that has locked down San Vanelona and placed the best skate spots off-limits. You’ll have to take back the skate scene with new tricks, fresh challenges, and all-new contests. Released during a long Tony Hawk’s dry spell, Skate 2 propelled the series to the throne of the skateboarding video game world.

You can download Skate 2 on the Xbox 360, but if you don’t have one, you’re stuck with emulation again.

5. OlliOlli 2: Welcome to Olliwood (2015)

The first OlliOlli broke away from the very successful formula developed by THPS and Skate. Instead of a 3D world packed with thousands of obstacles to skate and a seemingly infinite number of tricks and combos, it stood on simple 2D graphics and difficult, addicting gameplay. Gone are the days of throwing 1080 kickflip-to-indys. Land a simple 50-50 early in the game, and you’ll be elated.

The game keeps you on your toes by throwing new levels with surprising obstacles at you. And with gorgeous art, creative level design, and an almost unmatchable dose of adrenaline when you land the trick you’ve been attempting for the past hour, it’s really hard to put down. Don’t let the simple design fool you: you’ll sink many hours into this one.

And finally, this is a game you can easily buy and play on modern consoles. It’s available to download on your PlayStation 4 or Xbox One for a handful of dollars. It’s also a cross-play on PS4 and Vita, as are these other awesome cross-buy PlayStation titles.

Buy OlliOlli 2: Welcome to Olliwood on Steam.

6. Thrasher Presents Skate and Destroy (1999)

The game with the unfortunate task of releasing after the meteoric success of Tony Hawk’s Pro Skater, Thrasher Presents Skate and Destroy is a cult classic skateboarding game (and is also more commonly referred to as “Thrasher Skate and Destroy”). Where THPS focused on arcade skateboarding with flashy tricks and insane grind combos, Thrasher attempted to create realism between tricks and the player environment.

In that, Thrasher laid the groundwork for games like Skate, where the emphasis is on landing a trick to perfection and finding unique lines throughout a skatepark, instead of a button-bashing point-scoring trick-em-up. Not to say that Thrasher didn’t have a focus on high scores. You must accumulate enough points to rise through the 12 levels, dodging police officers, security guards, and more along the way.

Also, you cannot talk about Thrasher without mentioning the soundtrack, which features 13 of the finest hip-hop tracks ever produced.

Like the other options on this list, you’ll have to use an emulator to delve back into Thrasher. Check out how to download and install a PlayStation 1 emulator if you want to start skating again.

7. Skate or Die! (1987)

Skate or Die! is a classic skateboarding game originally released for the ZX Spectrum, Commodore 64, Atari ST, and other hardware of the era. It received a port to the NES where it sold over 100,000 copies and, with its catchy chiptunes, planted itself firmly in skateboard game history.

In Skate or Die!, you have the choice of five events: a freestyle ramp, high jump ramp (a half pipe), a downhill race, a downhill jam, and a jousting event (which is as it sounds). Although the tricks are basic, the Skate or Die! gameplay is fun, especially if you have someone else to play with.

If you don’t have a NES kicking about at home, check out how to build a custom Raspberry Pi NES or SNES emulator using RetroPie.

8. Skater XL (2020)

Skater XL is one of the newest skateboarding games about, offering an open-world sandbox on PC, PlayStation 4, Xbox One, and Nintendo Switch users. Developers Easy Day Studios are the team behind the massively popular Skater XL smartphone game and, realizing the popularity and demand for a more substantive game, have brought Skater XL to more powerful hardware.

The difference in hardware shows, as you might expect. Skater XL uses a similar control scheme to Skate and uses input from real-world tricks. That means practice makes perfect, and you’ll pick up and learn more tricks as you explore the Skater XL world. Adding to that, the Skater XL world is filled with awesome spots, including some real-world locations that mimic their counterparts, such as downtown Los Angeles.

At the time of writing, Skater XL is available on Steam Early Access and is scheduled to launch in the summer of 2020.

Buy Skater XL on Steam.

And One to Avoid: Tony Hawk’s Pro Skater 5

Tony Hawk’s Pro Skater 5 is terrible, and we would strongly advise you to avoid it. Buggy, laggy gameplay, and no innovation within the game itself make THPS5 one of the worst entries in the series. An online hub, missions for skaters, strange decisions regarding the special bar and trick limitations, and some lifeless level design combine to assign this title to the scrap heap.

The Best Skateboarding Games of All Time

For us, Skate 2 remains the best skateboarding game of all time. It hit that sweet spot of freedom, sandbox play, a semi-storyline to guide you through, and heaps of tricks and spots to try out.

Looking to the future, Skater XL looks like it could ascend the best skateboarding game throne, knocking all other competitors out of the way. Skater XL isn’t the only skateboarding game on the horizon either, with the almost fiendishly difficult Session also under development and available in Steam Early Access.

If you’re looking for some new games that aren’t skateboarding related, check out the best free PC games you can play right now.

Read the full article: The 8 Best Skateboarding Games of All Time

Read Full Article

10 Common Facebook Problems and Errors (And How to Fix Them)

Even though nearly everyone uses Facebook, there’s actually plenty wrong with Facebook.

For every great story or picture you see on Facebook, you have to put up with a dozen low-quality memes, annoying posts, and even frustrating user interface elements.

In this article, we’ll help you put a stop to that for good. We’ll cover solutions for some common Facebook problems so you don’t have to put up with them any more.

1. Mute Excessive Posters

We all know someone who posts all of the time on Facebook:

• An uncle shares too many political rants.
• Your grandpa has no idea how to use Facebook and posts nonsense every day.
• That one friend takes too many pictures of their pet.
• A friend from college copies and pastes a blurry JPEG riddled with grammatical errors to let you know that they really hate cancer.

These scenarios result in a Facebook friend dilemma. You don’t want to unfriend these people and hurt their feelings, but you’re also sick of seeing their posts. The solution is to unfollow them; you won’t see their updates any more, but will still remain friends.

To unfollow someone, visit their profile page. On the right side of their cover photo, you should see a Following box. Click that, and select Unfollow [Name].

Now, you won’t have to suffer from that person’s posts anymore. You’ll still be friends, so you can visit their timeline anytime if you want to check out what they’re up to. And if you change your mind, just click the Follow button to start seeing their posts again.

Unfollowing someone doesn’t send them a notification, so don’t worry about them knowing.

2. Keep Important Friends First

On the opposite end of the spectrum are friends whose posts you don’t want to miss. Facebook’s always-changing algorithms mean you might not even see certain updates from friends. To combat this, you can mark your closest friends out to always see their updates at the top of your feed.

Visit the friend’s page whose posts you can’t miss. Click the Following box again, but this time select See First. Now Facebook will always prioritize new updates from that person when you refresh your feed.

If you really can’t miss an update from a certain someone, you can get notifications every time they post. Visit their page, then click the Friends box. Select Get Notifications to have Facebook send you a notification every time that friend posts.

If you like, you can also add that person to the Close Friends group. Doing so lets Facebook know you care more about what they post, and you can also share posts exclusively with your Close Friends group.

3. Opt Out of Targeted Ads

Facebook makes most of its money from advertising. Thus, the platform spends a lot of time trying to serve you the most relevant ads in the hopes you’ll interact with them. By default, Facebook tracks you everywhere you go and uses your browsing habits to come up with the best ads for you.

You can opt out of these personalized ads and see more general ones for your age group and gender. To do so, click the small dropdown arrow at the top-right of Facebook’s interface and select Settings. Select Ads on the left sidebar to open a new page with ad preferences.

Expand the Ad settings category here, and you’ll see three categories. Disable Ads based on data from partners and Facebook won’t use your activity on other sites to show ads on Facebook. Turning off Ads based on your activity… will prevent your Facebook activity from being used to show you ads on other websites.

Finally, the Ads that include your social actions option prevents Facebook from using pages that you like to show ads to your friends.

To further reduce the information Facebook uses for ad purposes, expand Your information and disable each slider. This prevents Facebook from showing you ads based on your relationship status, education, and other factors.

4. Stop Sharing Posts With Everyone

Just like you get sick of seeing others’ posts, sometimes you don’t want to share your own updates with certain friends. Maybe you don’t want your boss seeing what you were up to on your day off, or don’t care for another round of commentary on your life from your sister. Using a few methods, you can easily hide your posts from certain people.

If you only want to hide a post from someone once, click the audience selector button (it probably says Friends or Public) on the post. Select Friends except and search for the friends you don’t want to see the post. Click the red Remove button to keep them from seeing the post, then Save Changes to finish.

To add keep someone from seeing your posts all of the time, you can add them to your Restricted list. People on this list won’t see anything you post unless it’s set to Public or you tag them in it. Visit someone’s page, select the Friends button, and then click Add to another list. Scroll down and click the Restricted list.

5. Disable Facial Recognition

Photo tagging is a fun feature on the surface, but it’s also a privacy risk. If someone takes an embarrassing picture of you and tags you in it, all of your friends will be able to see that picture. There’s always the risk that someone could use a tagged photo to obtain too much personal information about you. Plus, Facebook recognizing your face is a little creepy on its own.

You can’t completely block photo tags on Facebook, though you can prevent others from seeing tags. Go to Settings > Timeline and Tagging and change the Who can see posts you’re tagged in on your timeline? to Only me.

If it seems like Facebook tagging isn’t working, it could be that your friend disabled this option on their account. Try tagging someone else and see if that works, or refer to section #10 for tips on troubleshooting deeper Facebook errors.

This is overkill most of the time, so you may prefer to disable facial recognition instead. To do this, visit Settings > Face Recognition. Change the only setting under Face Recognition here to No.

After you do this, your friends won’t see tag suggestions like these any more:

6. Block Annoying Auto-Playing Videos

Very few people likes videos that play as soon as you scroll past them. They’re loud and can catch you off-guard if you thought you had your volume muted. Plus, if you’re on a metered connection, they’re a potential waste of data.

To turn off autoplay videos, click the top-right arrow anywhere on Facebook and choose Settings. Click the Videos tab on the left, and then set the Auto-Play Videos option to Off.

7. Block Annoying App Invites

Facebook features hundreds of stupid games, all of them designed to waste your time and take your money. If you’re sick of friends pleading for extra lives in Candy Crush Saga, you can block all Facebook invites and requests by game or by person.

Take another trip to Settings and pick the Blocking tab this time. Find the Block app invites field and enter any friend who’s invited you too many times. You’ll automatically block all game requests in the future. This won’t affect any other part of your interaction with them on Facebook.

If you’d like, you can also enter an app’s name in the Block apps field. This will prevent it from contacting you and using your Facebook information.

8. Change the Interface to Your Liking

You can’t change many elements of Facebook through the default options. If you want to take your Facebook customization to the next level, you’ll need to install a Facebook browser extension.

The best one is Social Fixer. It’s available as a browser extension for Chrome, Firefox, Safari, and Opera. Once you install Social Fixer, it makes several improvements out of the box. To tweak how it runs, click the wrench icon that appears in the upper-right corner of every Facebook page. Choose Social Fixer Options on the resulting menu to customize your Facebook experience.

Social Fixer offers too many options to cover here, but the highlights include:

• Automatically enable Most Recent view in your News Feed (General tab).
• Use Ctrl + Enter instead of Enter to submit comments (General tab).
• Enable the Friend Manager to get alerts when people unfriend you (General tab).
• Use the Hide Posts tab to hide any updates with certain words.
• Try some pre-made Filters to weed out politics, spoilers, and more.

Dig into the Social Fixer options and you’ll find many more ways to change the most annoying parts of Facebook.

Solutions for Common Facebook Errors

We conclude with tips for common Facebook errors people deal with when trying to access the social network.

9. You Forgot Your Facebook Password

Forgetting your Facebook login information is one of the biggest issues users run into. We’ve written a full guide to recovering your Facebook account if you can’t log in. And if you think your Facebook account was hacked, you need to act quickly.

10. You Can’t Connect to Facebook

Since Facebook is one of the biggest sites on the web, it rarely experiences an outage. So, the next time you can’t get on Facebook, follow a few steps to diagnose the problem:

1. Check a website like Down for Everyone to see if Facebook is really down. If it is, you can’t do much more than wait.
2. Visit another website to make sure your connection is working. If not, follow our steps to fixing internet connection problems.
3. Clear your browser cache and cookies, or try to open Facebook in an incognito window or another browser. If this works, there may be something wrong with your current browser.
4. Disable any VPNs or tracker-blocking extensions you have running and try to connect to Facebook directly.
5. Reboot your PC and router.

These tips also come in handy for strange glitches, like when the “see more” button is not working on Facebook. The chances are there’s a problem with your browser or internet connection. Try rebooting your PC and signing into Facebook with another browser when this happens.

How to Fix Facebook Problems Easily

We’ve shared fixes for some of the most common Facebook problems and errors. By flipping the right settings or using powerful extensions, you can make Facebook a more pleasant experience for yourself.

And for more help using the social network, check out our list of Facebook search tips.

Image Credits: Cienpies/Depositphotos, Artistan/Depositphotos

Read the full article: 10 Common Facebook Problems and Errors (And How to Fix Them)

Read Full Article

8 Awesome Solar-Powered Gadgets Every Home Should Be Using

Solar power provides cleaner, more sustainable electricity. But few people can afford to mount solar panels on their houses, and few power providers have invested significantly in solar power generation. So it’s time to take solar power into your own hands.

We’re highlighting great solar-powered devices that will help reduce your power bill and reduce reliance on fossil fuels.

1. YELOMIN Portable Solar Power Bank

YELOMIN Portable Solar Power Bank YELOMIN Portable Solar Power Bank Buy Now On Amazon $25.99

Your smartphone will always have a charge with the YELOMIN Portable Solar Power Bank. The 20,00mAh cell can completely recharge a smartphone three to five times depending on the model. There are two USB output plugs so you can charge two phones simultaneously.

When the power bank is depleted, just set in near a window to recharge. LED lights will let you know when the solar panel is working and the battery’s status. As a nice touch, the power bank also sports a flashlight with three different modes.

2. Soliom S60 Solar Outdoor Security Camera

Soliom S60 Solar Outdoor Security Camera Soliom S60 Solar Outdoor Security Camera Buy Now On Amazon $99.00

Solar power can help protect your home when you’re using the Soliom S60 Solar Outdoor Security Camera. The completely wireless camera is made to withstand extreme temperature swings and is waterproof.

When the camera detects motion, you’ll receive a notification on your smartphone thanks to the companion app. The camera will also record a video clip and store it on a memory card and store it in the cloud. At night, built-in IR LED lights will help brighten up what’s happening outside.

To find more about the camera, make sure to take a look at our Solim S60 review.

3. LECLSTAR Solar Motion Sensor Outdoor Light

LECLSTAR Solar Motion Sensor Outdoor Light LECLSTAR Solar Motion Sensor Outdoor Light Buy Now On Amazon $21.99 ($2.29 / oz)

Even if you’re not interested in a surveillance camera, solar power can still help provide extra security for your home. The LECLSTAR Solar Motion Sensor Outdoor Light is easy to mount with the included sticky pad or two screws. There’s no need to worry about wires or anything else as the light will recharge during the day.

Waterproof and made to withstand all types of weather, the light features 268 bright LEDs that provide up to 1,800 lumens of light. The built-in motion sensor can detect movement up to 26 feet away and with a 120-degree field and will automatically turn on the light for 30 seconds. When motion is no longer detected, it will automatically shut off.

4. ACOPOWER 150Wh Portable Solar Generator With 50W Solar Panel Combo

ACOPOWER 150Wh Portable Solar Generator With 50W Solar Panel Combo ACOPOWER 150Wh Portable Solar Generator With 50W Solar Panel Combo Buy Now On Amazon $247.99

Even generators are making the switch to a solar-powered future. The ACOPOWER 150Wh Portable Solar Generator with 50W Solar Panel Combo is a great way to harness solar power around the house. The generator sports two USB output plugs, two AC outlets, and three DC ports that can adapt to different equipment. While it won’t power large home electronics, it can provide enough juice to charge electrics and operate smaller appliances like a fan or lamp.

An included solar panel can completely charge the generator and its 40,800mAh battery in six to eight hours and folds up when not in use. The generator can also charge in a car or through a regular wall socket.

5. Logitech K750 Solar Wireless Keyboard

Logitech K750 Solar Wireless Keyboard Logitech K750 Solar Wireless Keyboard Buy Now On Amazon $47.47

Your computer already draws a lot of power—why use even more to power your accessories? Logitech’s K750 Solar Wireless Keyboard charges from the sun or artificial light and will keep a charge for up to three months in total darkness.

Measuring in at just 1/3-inch thick, it will be at home on any desk. And there’s no need to worry about wires; just plug in a small receiver into any free USB port. The keyboard is compatible with Windows 10 all the way through to Windows XP.

6. Goal Zero Torch 250 Flashlight

Goal Zero Torch 250 Flashlight Goal Zero Torch 250 Flashlight Buy Now On Amazon $69.95

A flashlight is another essential tool for any house. But it always seems that whenever you need one, the batteries are missing or out of power. That won’t be a problem with the Goal Zero Torch 250 Flashlight. The flashlight takes power from a built-in solar panel. And even if you can’t get near the sun, there is a hand crank or USB cord to use as a backup.

You can select from three different options on the 250-lumen flashlight—flashlight, floodlight, or red emergency light. On a full charge, the flashlight can run for up to 48 hours. You can also charge a smartphone using the built-in USB plug.

7. Eton Rugged Rukus Solar Powered Wireless Bluetooth Speaker

Eton Rugged Rukus Solar Powered Wireless Bluetooth Speaker Eton Rugged Rukus Solar Powered Wireless Bluetooth Speaker Buy Now On Amazon $74.72

Though it won’t give you the sound quality of a full-on speaker system, the Eton Rugged Rukus Solar Powered Wireless Bluetooth Speaker is highly portable and stores enough power to play music for eight hours on a full charge. The solar panel fully charges the speaker in less than five hours of direct sunlight.

It will also charge your smartphone via the USB port. The speaker is water-resistant as well, making it great for poolside listening and social events.

8. Lattis Ellipse Keyless Smart Solar Bike Lock

Lattis Ellipse Keyless Smart Solar Bike Lock Lattis Ellipse Keyless Smart Solar Bike Lock Buy Now On Amazon

The Lattis Ellipse Keyless Smart Solar Bike Lock is made to keep your bike safe and secure. The lock is made with 17mm forged steel and features a dual locking mechanism to fend off thieves.

If someone tries to tamper with the bike lock, you’ll receive an alert via Bluetooth on your smartphone. And when you’re ready to remove the lock, just place your smartphone near it, and it will automatically unlock. In case your phone isn’t available, there is a capacitive touchpad to enter a code as a backup.

Those smart features take power from a built-in solar panel that will constantly charge the lock under normal use. In an emergency, you can also charge it via the built-in micro-USB port.

Great Solar Gadgets for Your Home

Putting solar panels on your roof is really expensive and environmentally unsuitable in many locations. But that doesn’t mean you can’t start powering your home with solar energy. Using the devices above, you can charge your phone, light up the dark, listen to music, and even secure your bike with solar power. And that’s a great first step to a cleaner world.

Solar-powered devices are also great to use during power disruptions. Make sure to take a look at our list of other gadgets to protect yourself in case of a natural disaster.

Read the full article: 8 Awesome Solar-Powered Gadgets Every Home Should Be Using

Read Full Article

12 Little-Known Microsoft OneNote Features You Will Love

Microsoft OneNote is a personal information management app designed to help you take notes of all kinds into one digital notebook. You can organize your notes in any way you want, share them with others, and sync the content across devices, so they’re always at your fingertips. But the best thing about OneNote is that it’s packed full of features.

If you’re already using OneNote for Windows or OneNote 2016, then these features will speed up your work. And if you’ve never used it before, you’re about to find out how useful these OneNote features can be to organize almost any aspect of your life, work, and projects.

1. Sticky Notes

Sticky Notes creates virtual Post-it notes that you can stick anywhere on your screen. You can jot down quick notes, to-do items, paste in links or phone numbers, or store any snippets of text you need to remember. The app is not only deeply integrated with the OneNote app on Windows but also syncs your notes with all the devices.

Sticky Notes is a built-in app for Windows 10. If you can’t find Sticky Notes on your PC, you can install the app free from the Microsoft Store. On all your devices, you can create new notes, format them, add bullet points, or even insert pictures from a computer or camera.

2. Page Templates

When you develop a proper note-taking structure, you can start focusing on the content and your ideas. In the same way, when you create a template from scratch, it not only give pages a consistent look, it also provides flexibility in structuring different types of documents and saves time.

You can use OneNote templates in recurring projects to organize notes in a particular way. If you’re the minute taker for office meetings, you can manage attendees, conversations, and key points in a fixed tabular format.

Check out our guide to page templates and some of the best sites to download templates for use in different projects.

3. Custom Tags

OneNote’s system of organization with a typical notebook, section, and pages approach is excellent for organizing notes. But you can be more specific with tags.

They add a visual marker to help identify the type of information and contextual relationship across different notebooks or sections.

OneNote comes with a bunch of default tags (e.g., Important, To-Do, Question, etc.). But you can also create as many custom tags you want. Just remember that it should be specific, widely used across all your projects, and of high value.

Read this piece to find out how to use OneNote tags and why you should use them.

4. Internal Links

One of the most powerful OneNote features is the ability to make connections with other notes. Its deep linking structure ensures that you quickly engage in knowledge construction, critical thinking, and contextual learning.

To create a link, highlight a text or paragraph, right-click and select Link to Pages or Copy Link to Paragraph.

Then switch to another page and insert your link through the link dialog box. Alternatively, type [[, the name of the page verbatim, and then ]]. This will automatically convert it into an internal page link.

Check out our guide on how to create a personal wiki using OneNote.

5. Version History

Every time you make changes to a note, OneNote keeps track of the previous versions instead of completely overriding them. So if you delete any note by mistake, you can look through the page’s history to find the item you want.

In OneNote 2016, head to the History tab and click Page Versions.

You’ll see all the edits you made with their dates in the page navigation bar. And if you decide to restore that page, right-click the page or section and choose Restore Version.

Version histories consume OneDrive storage space. You can either delete the history of the different versions or disable them all together on a per-notebook basis.

6. Split and Merge Containers

One of the nice things about OneNote is that you can position content anywhere on the page. If you later decide that you want to organize them in a particular way, it quickly becomes a problem.

To pull out content from a container, select your text, then press and hold the left mouse button and drag the selection to a new position.

To merge the contents of two containers, hover your mouse to the three-dot title bar of the source container. A four-sided cursor will appear. Press and hold the Shift + Left-click key.

With the mouse button pressed, drag-and-drop the source container into the destination container. The contents will get automatically merged.

7. Translate Text or Entire Page

As communication at work is increasingly getting multilingual, the need to translate text in different languages is becoming essential. Microsoft provides you the powerful Office translation tools to decipher content in various languages and puts it into a note.

In the OneNote app, select a paragraph of text, right-click it and choose Translate.

For translating the entire page, navigate to the View tab, and choose Translate > Page.

A side panel appears with your native language at the top. From the bottom pane, select the required language from the dropdown menu. If you translate the entire page, OneNote will create a new translated version of the page.

8. Sort Pages in OneNote

Every time you add a new page, it gets added to the bottom of the list by default. Although you can tweak this option in Settings to create new pages At the top of the page list or Below the selected page, the ability to sort your pages with individual preferences cannot be ignored.

To sort your pages in OneNote, click the Sort button next to your notebook. Then choose any one option from the list — Alphabetical, Date Created, or Date Modified.

You can select the sorting order on a per-section basis. In OneNote 2016, use the Onetastic add-in and install the Sort Pages macro to sort your pages.

9. Quick-Formatting Mini Toolbar

OneNote includes a hovering toolbar called Mini Toolbar. It appears when you select a piece of text and want to take some relevant action. While you can ignore it and perform the actions through Quick Access Toolbar, you’re missing many minute features.

Through the mini toolbar, you can adjust your fonts, annotate the text, tag your content, and more. Click on the three-dot icon to further expand the toolbar with a context menu. You can edit your notes much faster with relevant tools.

10. Solve Math Equations With Math Assistant

If you need a little help with your math homework, the built-in math assistant can help you solve or display step-by-step instructions to learn how to approach a particular problem.

This feature is only available if you have an Office 365 subscription. Here are some ways to get Microsoft Office license for free.

In the Draw tab, write or type in your equation.

Use the Lasso Select tool to draw a circle around the equation.

Then select Math to open Math Assistant pane.

To solve the equation, click the Select an action box and choose an action.

Review the solution or click Show steps to learn more in detail. Generate a practice quiz to keep practicing similar types of math problems.

11. Email to OneNote

Apart from typing notes on an OneNote page, you can also send it by using the Email to OneNote service. It consists of an email address me@onenote.com to which you can forward email messages (trip confirmation, receipts, or important emails) or share any type of content.

The subject field of the email becomes the title of your OneNote page. Also, any link you include in the body of the message automatically gets expanded into an article and a full-page screenshot.

Apart from Outlook, you can even use Gmail or iCloud ID’s. You just have to set it up once.

12. Paste Text Without Formatting

When you copy and paste content, OneNote will paste the text along with its formatting and a source link whether you use the Ctrl + V shortcut or Paste option from the context menu.

If you don’t want the link and formatting garbage, having to choose Paste > Keep Text Only repeatedly becomes tedious.

To change the default paste settings, open OneNote and click on More > Settings in the top-right corner of the window.

Click Options, and just under the Paste Options section, choose Keep Text Only from the dropdown menu. And turn off the Include link to source option.

Unique Ways to Use OneNote

These little-known tips are some of the best features of OneNote. The app can be useful for so many things with these advanced features. To know more, read this piece to find out some unique ways to use OneNote to organize life and work projects.

Read the full article: 12 Little-Known Microsoft OneNote Features You Will Love

Read Full Article

Essential Facebook Etiquette: 10 Do’s and Don’ts

Given how popular Facebook is, you’d think everyone would know how to use it properly. Sadly, that simply isn’t the case.

While Facebook has many written rules, there are also plenty of unwritten Facebook rules. This is the Facebook etiquette we should all follow.

So, in this article, we look at some Facebook do’s and don’ts to help you become a more responsible user.

Facebook Etiquette: Do’s

First, let’s review some behavior that’s considered polite on Facebook.

1. Do Accept Requests From People You Know

Facebook is all about connecting with friends, old and new. Because of this, it’s a good idea to accept requests from people who want to connect.

You’ll also regularly see recommendations on people you might want to add, based on mutual friends or other shared connections. If you have trouble navigating this, read our guide to Facebook friend requests.

However, you should be aware of phony Facebook friend requests, which involve scammers making a fake account with the name of someone you know. If you receive a request from someone that you’re already friends with, you should reach out to them to make them aware of the fake.

2. Do Acknowledge Others’ Posts and Comments

It’s easy to scroll through everything you see on Facebook without engaging, but that kind of defeats the purpose. It’s polite to respond to people who comment on your post—simply thanking them for the compliment about your new haircut goes a long way.

And if you want to react without writing a comment, hover your mouse over the Like button to choose from several reactions, such as Cry or Laugh.

3. Do Engage in Healthy Discussions

When you come across something that a Facebook friend posted that you have an opinion about, feel free to have a healthy discussion about it. However, if it turns into a nasty argument, it’s better to leave it alone rather than risk ruining your friendship.

Even if you do get into a heated discussion with someone, you shouldn’t impulsively remove them from your friends list. You can just politely ignore people on Facebook for a while instead.

Try to remember that even in online discussions, you are talking to real people. It’s a good idea to keep your tone polite and respectful at all times. Keep in mind that while engaging with people who don’t necessarily agree with you can be rewarding, Facebook isn’t a particularly good medium for it.

4. Do Research Stories Before Sharing Them

Image Credit: rawpixel/Pixabay

Social media sites make it easy to spread misinformation a long way in no time at all. Before you share a link or a photo on Facebook, especially if someone else shared it, do a quick Google search or visit one of these fact-checking websites. If the story is known to be false, you can avoid spreading it.

You can further do your part by leaving a comment on false stories when someone shares them. Provide a link to a fact-checking article proving it false so others are informed.

By the way, this also applies to the “copy and paste” status updates that make their rounds on Facebook regularly. If you see a post telling you that you need to share an update or Facebook will limit your friends list, it isn’t legitimate.

5. Do Ask for Permission Before You Tag Someone

If you take photos when you’re out with your friends, it’s polite to ask for permission before you tag them in those photos. Once they’re tagged in a photo, all of their friends can see it.

Avoid sharing unflattering photos of your friends, which will probably upset them. The golden rule is to treat your friends like you’d want them to treat you.

Facebook Etiquette: Don’ts

Next, let’s look at some bad Facebook etiquette. You should avoid these actions to be a better citizen of the social network.

1. Don’t Send Friend Requests to Everyone

If you send someone a friend request after just a brief conversation, or maybe even without having talked to them in person, they might not be ready to add you. Be sure to only send friend requests to people who know you and would expect them.

Sending too many friend requests to people you don’t know will lead to people turning you down. They might even mark your request as spam, which could get you in trouble on Facebook if it happens too often.

On a similar note, here’s how to ask a girl out on Facebook if you’re looking to do so.

2. Don’t Post Inappropriate Photos

Facebook doesn’t allow outright nudity or sexually suggestive photos on the platform. However, even if your photo doesn’t break the rules, you should consider how the people on your friends list might react to it.

For example, if you’re friends with your 13-year-old niece on Facebook, you might want to reconsider posting about crazy parties where you passed out on the floor. If you really want to share such photos, create a private chat in Facebook Messenger and share them with close, age-appropriate friends there.

You might also consider using friend lists, available on the Explore tab on the left side of Facebook. Select Friend Lists, then you can make a new list and share with only them when you post an update.

3. Don’t Post Too Often

Each social media platform has its own etiquette when it comes to sharing. While it’s acceptable to post hundreds of times a day on Twitter, Facebook doesn’t work that way. People generally share a few interesting stories, photos, or life updates on their Facebook pages as they happen. Don’t spam your friends with dozens of memes or anecdotes every day.

If you have a lot you want to share, consider using a service like Buffer to schedule your posts at regular intervals.

4. Don’t Share Sensitive Content

Bragging about doing something illegal will not only make you look bad, but could also land you in legal trouble. Say that you drink and drive, then post about it on Facebook. Not only will this reveal you to be a lawbreaker, but someone could report you to the police for it.

There’s another element to this: don’t share too many personal details on Facebook. It’s not smart to share confidential info that someone could use against you.

If you’re going through a breakup, for instance, it’s not appropriate to share all of the gory details. Remember that anything you post online could be accessible forever, even if you think only your friends will see it.

5. Don’t Bully Others

Bullying isn’t just nasty; it can also be against the law depending on the context. Don’t make offensive comments about a person’s immutable characteristics on Facebook. Even just a few snarky words you write can harm others in a big way.

Cyberbullying can lead to serious issues, especially with young people. Keep your Facebook environment a friendly place where everyone is respected.

Treat Others As You Want to Be Treated

When it comes to posting on Facebook, remember that online communities are still filled with real people. Before you post something on Facebook, stop and think whether you would say or do that offline. While it’s good to be yourself and express your personality, doing so shouldn’t come at the expense of others.

To get more out of the social network, here’s how to fix some common Facebook problems.

Read the full article: Essential Facebook Etiquette: 10 Do’s and Don’ts

Read Full Article

Is Disney+ Worth Getting and Worth the Money?

These days, there are a lot of streaming services available to choose from. So much so that it’s difficult to decide which ones are worth getting and which ones represent good value for money. And that includes Disney+.

In this article, we’ll help you decide whether Disney+ is worth getting for the money. We’ll also compare Disney+—which offers access to movies and shows from Disney, Pixar, Marvel, Star Wars, and National Geographic—to other streaming services.

How Much Does Disney+ Cost?

At $6.99/month (in the US), Disney+ is one of the cheapest paid streaming services available. For that price, you can add seven profiles to your account and watch movies or TV shows on four devices at a time.

You can also sign up for a one year subscription with an upfront payment of $69.99. That’s the equivalent of $5.83/month, making Disney+ even cheaper.

If you aren’t ready to sign up to Disney+ just yet, you can try out Disney+ for yourself with a seven-day free trial.

Disney+ content is available to stream or download on almost every device you own, including your TV, smartphone, tablet, and laptop. And you don’t need to pay extra to unlock 4K video, which is the case for some of the best streaming TV services.

There are absolutely no adverts and the service is packed with thousands of hours of content.

How the Price of Disney+ Compares to Other Services

The biggest competitors to Disney+ are Netflix, Amazon Prime, and Hulu. Each of these services offers a wider range of plans, with certain features locked behind paywalls. Other streaming services also exist, like Apple TV+, but since Apple is limited to original content its library pales in comparison to the competition.

Netflix offers three subscription plans: Basic, Standard, and Premium. The Basic plan costs $8.99/month—which is already more expensive than Disney+—and only lets you watch standard definition videos on a single screen at a time.

Amazon also offers a basic Prime Video subscription for $8.99/month. But most people go for the $12.99/month option to get unlimited music streaming, photo storage, and reading as well. You can watch Prime Video movies or TV shows on three different screens at once. And since it’s Amazon Prime, you also get free two-day shipping.

Also owned by Disney, Hulu is the only option with a lower subscription fee at $5.99/month. That said, you can still get Disney+ cheaper by signing up for an entire year. You also need to pay more for Hulu if you want to unlock extra channels or get rid of the adverts.

What You Get With Disney+

Disney+ includes Pixar, Marvel, Star Wars, National Geographic, and Disney content. This collection covers an enormous number of TV shows alongside some of the best movies ever made.

We’ve already written about the best movies and TV shows to watch on Disney+. However, the highlights include Disney’s animated feature films, the Star Wars movies, and the entire Marvel Cinematic Universe.

Considering that many of these movies still sell for $20 each on the iTunes Store, getting access to all of them for $5.99/month makes Disney+ a pretty good deal.

Original Disney+ Content

Most of the movies you’re likely to watch on Disney+ were enormously successful releases. That means you’ve probably seen them several times already and you might even own them on DVD or Blu-ray. In the long run, you’re probably going to be more interested in Disney’s original content.

The Mandalorian is the biggest show on Disney+. And it’s possibly the only one worth talking about. There aren’t a huge number of original titles yet, and while many of them are enjoyable, none has taken the world by storm like The Mandalorian (mainly thanks to “Baby Yoda”).

Disney+ will also be the home to future Marvel and Star Wars live-action shows. These are expected to include The Falcon and the Winter Soldier, WandaVision, and more live-action Star Wars spinoffs. Many of these shows are likely to tie-in to future cinematic releases, meaning they will be essential viewing for a lot of fans.

Take note that Disney releases original episodes weekly. So if you want to binge an entire season in one month, you should wait until every episode is released before you sign up.

How the Content Library Compares to Other Services

Because Disney owns the rights to all of this content, Disney+ is likely to have a more stable library than other services. While movies frequently appear and disappear from Netflix, Amazon Prime, or Hulu, those that are on Disney+ are there to stay.

However, this stability comes at a cost: Disney’s library isn’t as broad or varied as other services. Most of the TV shows are from The Disney Channel and everything is family-friendly.

In contrast, other services give you fresh movies or TV shows all the time as they cycle content through the platform. You also get a wider range of viewing options, with Hulu offering everything from Family Guy to Teen Titans, for example.

What You Don’t Get With Disney+

Disney has a well-established brand to maintain, which means there is no mature content available on Disney+. Everything is rated PG-13 or lower, making it a perfect safe space for kids, but a little too sanitized for adults.

Don’t expect to find shows like Game of Thrones or The Walking Dead on Disney+ any time soon.

What’s more, each of the major franchises on Disney+ is better known for its movies than its TV shows. As a result, there’s a disappointing lack of quality TV shows for you to watch when you don’t have time to watch a full movie.

20th Century Fox Content

In 2019, Disney purchased 20th Century Fox. However, there isn’t much Fox content on the platform yet. It’s possible this is because Disney needs to maintain its family-friendly brand, which isn’t possible if it offers up Family Guy, Die Hard, and the Alien movies to stream.

At the moment, the only major Fox titles available on Disney+ are The Simpsons, Avatar, and some of the X-Men movies. Still, there’s no Deadpool and no Logan because those were both rated R.

It’s possible that Disney will filter more Fox content into its platform in the future. This would make Disney+ more worthwhile for adult subscribers. However, some speculators predict that that’s what Disney intends to use Hulu for instead.

Is Disney+ Worth Getting for the Money?

If you like the look of the movies and TV shows available to watch on Disney+ then it’s well worth the money. Purely in terms of the numbers, the subscription fee is cheap compared to a lot of other streaming services. And you get access to a lot of popular and highly-rated movies.

If you have children who might enjoy Disney Channel titles and Marvel cartoons, there’s a seemingly endless number of TV shows for them to watch as well. However, there aren’t a great deal of quality shows for adults outside of The Mandalorian and The Simpsons.

Hopefully that changes in the future as Disney releases more originals and/or introduces more 20th Century Fox content. But only time will tell.

To summarize, Disney+ is absolutely worth getting if you want to watch Pixar, Star Wars, Marvel, and Disney movies, plus some interesting documentaries courtesy of National Geographic. There are also plenty of classic films worth watching on Disney+.

Read the full article: Is Disney+ Worth Getting and Worth the Money?

Read Full Article

An Optimistic Perspective on Offline Reinforcement Learning

Posted by Rishabh Agarwal, AI Resident and Mohammad Norouzi, Research Scientist, Google Research

“The potential for off-policy learning remains tantalizing, the best way to achieve it still a mystery.” — Sutton & Barto

Most reinforcement learning (RL) algorithms assume that an agent actively interacts with an online environment to learn from its own collected experience. These algorithms are challenging to apply to complex real-world problems (such as robotics and autonomous driving) since extensive data collection from the real world can be extremely sample inefficient and lead to unintended behavior, while those operating in simulation require high-fidelity simulators that are challenging to build. However, for many real-world RL applications, there already exist a large amount of previously collected interaction data which can be utilized to make RL feasible for those problems, and enable better generalization by incorporating diverse prior experiences.

Existing interaction data can be used effectively using offline RL, which is the fully off-policy RL setting in which an agent is trained from a fixed dataset of logged experiences, without any further interactions with the environment. Offline RL can help (1) pretrain an RL agent using existing data, (2) empirically evaluate RL algorithms based on their ability to utilize a fixed dataset of interactions, and (3) deliver real-world impact. However, offline RL is considered challenging due to the distribution mismatch between online interactions and any fixed dataset of logged interactions, i.e., when the learned agent takes an action different from the data collection agent, we don’t know the reward that should be provided.

RL with online interactions vs. Offline RL.

In “An Optimistic Perspective on Offline RL”, we propose a simple experimental setup for offline RL on Atari 2600 games, based on logged experiences of a DQN agent. We demonstrate that it is possible to train agents with high returns that outperform the data collection agents using standard off-policy RL algorithms, without explicitly correcting for any distribution mismatch. We also develop a robust RL algorithm, called random ensemble mixture (REM), which shows promising results on offline RL. Overall, we present an optimistic perspective that robust RL algorithms trained on sufficiently large and diverse offline datasets can lead to high quality behaviour, strengthening the emerging data-driven RL paradigm. To facilitate the development and evaluation of offline RL methods, we are also publicly releasing the DQN Replay Dataset and have open-sourced our code. More details can be found at offline-rl.github.io.

A Primer on Off-policy and Offline RL
We summarize various approaches to RL below:

Online, off-policy RL agents, such as DQN, achieve human-level performance on Atari 2600 games by just observing the game screen, without any explicit knowledge about the game. DQN estimates the effectiveness of an action at a given state of the environment in terms of maximum achievable future rewards (i.e., Q-values). Furthermore, recent distributional RL agents, such as QR-DQN, model the entire distribution of probable future rewards, rather than a single expected value for each state-action pair. Agents such as DQN and QR-DQN are considered “online” because they alternate between optimizing a policy (how an agent acts at a given state) and using that policy to collect more data.

In principle, off-policy RL agents can learn from data collected by any policy, not just the policy being optimized. However, in the offline RL setting, recent work presents a discouraging view that standard off-policy agents diverge or otherwise yield poor performance. To fix this, previous work proposes remedies by regularizing the learned policy to stay close to the dataset of offline interactions.

The DQN Replay Dataset for Offline RL
In this work, we revisit offline RL by first creating the DQN Replay Dataset. This dataset is generated using DQN agents trained on 60 Atari 2600 games for 200 million frames each, while using sticky actions (with 25% probability that the agent’s previous action is executed instead of the current action) to make the problem more challenging. For each of the 60 games, we train 5 DQN agents with different random initializations, and store all of the (state, action, reward, next state) tuples encountered during training into 5 replay datasets per game, resulting in a total of 300 datasets.

Offline RL on Atari games using the DQN Replay Dataset.

The DQN Replay Dataset can then be used for training offline RL agents, without any interaction with the environment during training. Each game replay dataset is approximately 3.5 times larger than ImageNet and includes samples from all of the intermediate policies seen during the optimization of online DQN.

Training Offline Agents on the DQN Replay Dataset
We trained offline variants of DQN and distributional QR-DQN on the DQN Replay Dataset. Although the offline datasets contain data experienced by a DQN agent improving over time as training progresses, we compared the performance of offline agents against the best performing online DQN agent obtained after training (i.e., a fully-trained DQN). For each game, we evaluated the 5 offline agents trained (one per dataset), using online returns, reporting the best averaged performance.

Offline DQN underperforms fully-trained online DQN on all except a few games, where it achieves higher scores with the same amount of data. Offline QR-DQN, on the other hand, outperforms offline DQN and fully-trained DQN on most of the games. These results demonstrate that it is possible to optimize strong agents offline using standard deep RL algorithms. Furthermore, the disparity between the performance of offline QR-DQN and DQN indicates the difference in their ability to exploit offline data.

Offline DQN. Normalized improvement over a fully-trained DQN, per game, of offline DQN trained using DQN replay. On the normalized scale, fully-trained DQN corresponds to 100% performance while random agent corresponds to 0%.

Offline QR-DQN. Normalized performance improvement (in %) over a fully-trained DQN agent, per game, of offline QR-DQN trained offline using DQN replay.

Introducing Two Robust Offline RL Agents
In online RL, an agent chooses actions that it thinks will lead to high rewards, and then receives corrective feedback. Since it is not possible to collect additional data in offline RL, it is essential to reason about generalization using a fixed dataset. Leveraging methods from supervised learning that use an ensemble of models to improve generalization, we present two new offline RL agents:

• Ensemble-DQN is a simple extension of DQN that trains multiple Q-value estimates and averages them for evaluation.
  
• Random Ensemble Mixture (REM) is an easy to implement extension of DQN inspired by Dropout. The key intuition behind REM is that if one has access to multiple estimates of Q-values, then a weighted combination of the Q-value estimates is also an estimate for Q-values. Accordingly, in each training step, REM randomly combines multiple Q-value estimates and uses this random combination for robust training.

Neural Network architectures for DQN, distributional QR-DQN and the expected RL variants with the same multi-head QR-DQN architecture, i.e., Ensemble-DQN and REM. In QR-DQN, each head (red rectangles) corresponds to a specific fraction of the return distribution, while in the proposed variants, each head approximates the Q-function.

To utilize the DQN Replay Dataset more efficiently, we train offline agents for five times as many training iterations as online DQN and report their performance below. Offline REM outperforms offline DQN and offline QR-DQN. The comparison with fully-trained online C51, a strong distributional agent, illustrates that the gains from offline REM are more than the gains from C51.

Offline REM vs. baselines. Median normalized scores averaged over 5 runs across 60 Atari games of offline agents trained using DQN replay for 5x iterations, compared to online DQN.

Using the standard training protocols on Atari, online REM performs on par with QR-DQN in the standard online RL setting. This suggests that we can use the insights gained from the DQN Replay Dataset and the offline RL setting to build effective online RL methods.

Online REM vs. baselines. Median normalized evaluation scores averaged over 5 runs (shown as traces) across stochastic 60 Atari 2600 games of online agents trained for 200 million frames. Online REM with 4 Q-networks performs comparably to online QR-DQN.

Comparison of Results: Important Factors in Offline RL
The discrepancy between these results and prior work that reports failure of standard RL agents in the offline setting could be attributed to the following factors:

• Offline Dataset Size. We trained offline QR-DQN and REM with reduced data obtained via randomly subsampling the entire DQN Replay Dataset, maintaining the same data distribution. Analogous to supervised learning, performance tends to increase as the size of data increases. With only 10% of the entire dataset, REM and QR-DQN approximately recover the performance of fully-trained DQN.
• Offline Dataset Composition. We trained offline RL agents on the first 20 million frames per game in the DQN Replay Dataset. Offline REM and QR-DQN outperform the best policy in this lower quality dataset, indicating that standard RL agents work well in the offline setting with sufficiently diverse datasets.
  

  

  Offline RL with Lower Quality Dataset. REM and QR-DQN trained on offline data collected from DQN trained for 20 iterations (by using the first 20M frames from each game replay dataset). The horizontal line shows the performance of best policy in this dataset, which is significantly worse than fully-trained DQN.

• Offline Algorithm Choice. There are claims that standard off-policy agents are ineffective on continuous control tasks when trained offline. However, we found that recent continuous control agents, such as TD3, perform comparably to a sophisticated offline agent when trained on large and diverse offline datasets.

Future Work. Our results emphasize the need for a rigorous characterization of the role of generalization due to neural networks when learning from offline data collected from a large mixture of diverse policies. Benchmarking offline RL with various data collection strategies by subsampling DQN replay (e.g., first / last k million frames) is another important direction. We currently employ online policy evaluation, however, “true” offline RL requires offline policy evaluation for hyperparameter tuning and early stopping. Finally, model-based RL and self-supervised learning approaches are also promising for offline RL.

Acknowledgements.
This research was conducted in collaboration with Dale Schuurmans. We’d like to thank members of the Google Research, Brain Team for valuable discussions. A prior version of this work was presented as a contributed talk at NeurIPS 2019 DRL Workshop.