Read Full Article
A blog about how-to, internet, social-networks, windows, linux, blogging, tips and tricks.
07 November 2017
Tests give iPhone X display top honors, but camera is merely competitive
Read Full Article
Tests give iPhone X display top honors, but camera is merely competitive
Read Full Article
Boxed shows off the automated tech it uses to ship toilet paper and more
Read Full Article
Tangent: Source-to-Source Debuggable Derivatives
Posted by Alex Wiltschko, Research Scientist, Google Brain Team
Tangent is a new, free, and open-source Python library for automatic differentiation. In contrast to existing machine learning libraries, Tangent is a source-to-source system, consuming a Python function f and emitting a new Python function that computes the gradient of f. This allows much better user visibility into gradient computations, as well as easy user-level editing and debugging of gradients. Tangent comes with many more features for debugging and designing machine learning models:
- Easily debug your backward pass
- Fast gradient surgery
- Forward mode automatic differentiation
- Efficient Hessian-vector products
- Code optimizations
Neural networks (NNs) have led to great advances in machine learning models for images, video, audio, and text. The fundamental abstraction that lets us train NNs to perform well at these tasks is a 30-year-old idea called reverse-mode automatic differentiation (also known as backpropagation), which comprises two passes through the NN. First, we run a “forward pass” to calculate the output value of each node. Then we run a “backward pass” to calculate a series of derivatives to determine how to update the weights to increase the model’s accuracy.
Training NNs, and doing research on novel architectures, requires us to compute these derivatives correctly, efficiently, and easily. We also need to be able to debug these derivatives when our model isn’t training well, or when we’re trying to build something new that we do not yet understand. Automatic differentiation, or just “autodiff,” is a technique to calculate the derivatives of computer programs that denote some mathematical function, and nearly every machine learning library implements it.
Existing libraries implement automatic differentiation by tracing a program’s execution (at runtime, like TF Eager, PyTorch and Autograd) or by building a dynamic data-flow graph and then differentiating the graph (ahead-of-time, like TensorFlow). In contrast, Tangent performs ahead-of-time autodiff on the Python source code itself, and produces Python source code as its output.
As a result, you can finally read your automatic derivative code just like the rest of your program. Tangent is useful to researchers and students who not only want to write their models in Python, but also read and debug automatically-generated derivative code without sacrificing speed and flexibility.
You can easily inspect and debug your models written in Tangent, without special tools or indirection. Tangent works on a large and growing subset of Python, provides extra autodiff features other Python ML libraries don’t have, is high-performance, and is compatible with TensorFlow and NumPy.
Automatic differentiation of Python code
How do we automatically generate derivatives of plain Python code? Math functions like tf.exp or tf.log have derivatives, which we can compose to build the backward pass. Similarly, pieces of syntax, such as subroutines, conditionals, and loops, also have backward-pass versions. Tangent contains recipes for generating derivative code for each piece of Python syntax, along with many NumPy and TensorFlow function calls.
Tangent has a one-function API:
Here’s an animated graphic of what happens when we call tangent.grad on a Python function:
If you want to print out your derivatives, you can run:
Under the hood, tangent.grad first grabs the source code of the Python function you pass it. Tangent has a large library of recipes for the derivatives of Python syntax, as well as TensorFlow Eager functions. The function tangent.grad then walks your code in reverse order, looks up the matching backward-pass recipe, and adds it to the end of the derivative function. This reverse-order processing gives the technique its name: reverse-mode automatic differentiation.
The function df above only works for scalar (non-array) inputs. Tangent also supports
- Using TensorFlow Eager functions, for processing arrays of numbers.
- Subroutines
- Control flow
Next Steps
Tangent is open source now at http://ift.tt/2j4jW8b. Go check it out for download and installation instructions. Tangent is still an experiment, so expect some bugs. If you report them to us on GitHub, we will do our best to fix them quickly.
We are working to add support in Tangent for more aspects of the Python language (e.g., closures, inline function definitions, classes, more NumPy and TensorFlow functions). We also hope to add more advanced automatic differentiation and compiler functionality in the future, such as automatic trade-off between memory and compute (Griewank and Walther 2000; Gruslys et al., 2016), more aggressive optimizations, and lambda lifting.
We intend to develop Tangent together as a community. We welcome pull requests with fixes and features. Happy deriving!
Acknowledgments
Bart van Merriƫnboer contributed immensely to all aspects of Tangent during his internship, and Dan Moldovan led TF Eager integration, infrastructure and benchmarking. Also, thanks to the Google Brain team for their support of this post and special thanks to Sanders Kleinfeld and Aleks Haecky for their valuable contribution for the technical aspects of the post.
Is Out-of-Date EFI Firmware Putting Your Mac at Risk?
When news breaks of a Mac security flaw, it almost always makes headlines. It should not be a surprise that a paper about Mac’s potential vulnerability to an EFI exploit became big news, even when the study’s authors say that most consumer users should not worry. Most EFI attacks require physical access and sophisticated tools. However, they do warn about the risk when crossing borders during travel. Corporate users that have to protect sensitive data are also at risk. If that sounds like you, how do you defend yourself? What Is EFI? You may be thinking: “I never downloaded EFI!...
Read the full article: Is Out-of-Date EFI Firmware Putting Your Mac at Risk?
Read Full Article
Google launches Resonance Audio, its new spatial audio SDK
Read Full Article
A tale of two chip stocks
Read Full Article
iPhone X first weekend adoption tops the iPhone 8 models
Read Full Article
Need a Gantt Chart Template for Excel or PowerPoint? Here Are 10 Unique Options
You don’t need Microsoft Project to create a Gantt chart for your project. If you’re just starting out with project management and do not have access to such software, you might just use Microsoft Excel or PowerPoint for tracking. If that sounds like you, then a Gantt chart template may be just what you need. We’ve compiled a selection of 10 Gantt chart templates for Excel and PowerPoint that can help you get your project off to a great start. 1. Detailed Gantt Project Planner (Microsoft Office) Microsoft Office has a terrific template that you can download from the website...
Read the full article: Need a Gantt Chart Template for Excel or PowerPoint? Here Are 10 Unique Options
Read Full Article
Photoslurp bags $870k for its ‘shop the look’ UGC marketing platform
Read Full Article
You Can Still Upgrade to Windows 10 for Free… If You’re Quick
If you haven’t yet switched to Windows 10, either by upgrading your current PC or by buying a new one, it’s not too late to do so for free. That is if you’re reading this before the end of 2017, when Microsoft has decided to end free Windows 10 upgrades for good. So, you’d best act quickly… Microsoft launched Windows 10 in July 2015, and, for exactly one year, Windows 10 was available for free. However, since July 2016 there has been a workaround allowing anyone to upgrade to Windows 10 for free. You have to claim to use assistive...
Read the full article: You Can Still Upgrade to Windows 10 for Free… If You’re Quick
Read Full Article
06 November 2017
B&O’s Beoplay E8 totally wireless earbuds really are the total package
Read Full Article
B&O’s Beoplay E8 totally wireless earbuds really are the total package
Read Full Article
Looking for the Best Programming Languages? Start Here!
Whether you’re completely new to programming or looking to jump into a new kind of programming you’ve never touched, one of the most important questions to ask is: “Which programming language is right for me?” These days, you have so many languages to choose from, it can be quite overwhelming to pick one when you don’t have much experience. In this article, we’ll cover the best modern programming languages for web, mobile, desktop, game, embedded, and utility development. Treat this as an overview and a jumping-off point. For Web Development Web development is unique in the programming realm, which presents...
Read the full article: Looking for the Best Programming Languages? Start Here!
Read Full Article
Google is surfacing Texas shooter misinformation in search results — thanks also to Twitter
Read Full Article
Google is surfacing Texas shooter misinformation in search results — thanks also to Twitter
Read Full Article
Have a Specific Purpose for Your PC? Use This Linux OS
Unlike Windows and macOS, Linux is what you make it. There are few limitations on which parts of the operating system you can swap out. So you’re free to install any Linux operating system and tweak it into your dream machine. But you could save time by installing a Linux operating system that already does what you want. Thing is, there aren’t just one or two Linux operating systems to choose from — there are hundreds. Each bundles together different free and open source components in different ways. How do you know which one to use? I can’t answer that...
Read the full article: Have a Specific Purpose for Your PC? Use This Linux OS
Read Full Article
Album+ organizes photos with A.I. that runs on your phone, not in the cloud
Read Full Article