Apple’s Voice Control improves accessibility OS-wide on all its devices

Apple is known for fluid, intuitive user interfaces, but none of that matters if you can’t click, tap, or drag because you don’t have a finger to do so with. For users with disabilities the company is doubling down on voice-based accessibility with the powerful new Voice Control feature on Macs and iOS (and iPadOS) devices.

Many devices already support rich dictation, and of course Apple’s phones and computers have used voice-based commands for years (I remember talking to my Quadra). But this is a big step forward that makes voice controls close to universal — and it all works offline.

The basic idea of Voice Control is that the user has both set commands and context-specific ones. Set commands are things like “Open Garage Band” or “File menu” or “Tap send.” And of course some intelligence has gone into making sure you’re actually saying the command and not writing it, like in that last sentence.

But that doesn’t work when you have an interface that pops up with lots of different buttons, fields, and labels. And even if every button or menu item could be called by name, it might be difficult or time-consuming to speak everything out loud.

To fix this Apple simply attaches a number to every UI item in the foreground, which a user can show by saying “show numbers.” Then they can simply speak the number or modify it with another command, like “tap 22.” You can see a basic workflow below, though of course without the audio cues it loses a bit:

Remember that these numbers may be more easily referenced by someone with little or no vocal ability, and could in fact be selected from using a simpler input like a dial or blow tube. Gaze tracking is good but it has its limitations, and this is a good alternative.

For something like maps, where you could click anywhere, there’s a grid system for selecting where to zoom in or click. Just like Blade Runner! Other gestures like scrolling and dragging are likewise supported.

Dictation has been around for a bit but it’s been improved as well; You can select and replace entire phrases, like “Replace ‘be right back’ with ‘on my way.’ ” Other little improvements will be noted and appreciated by those who use the tool often.

All the voice processing is done offline, which makes it both quick and robust to things like signal problems or use in foreign countries where data might be hard to come by. And the intelligence built into Siri lets it recognize names and context-specific words that may not be part of the base vocabulary. Improved dictation means selecting emoji and adding dictionary items is a breeze.

Right now Voice Control is supported by all native apps, and third party apps that use Apple’s accessibility API should be able to take advantage of it easily. And even if they don’t do it specifically, numbers and grids should still work just fine, since all the OS needs to know are the locations of the UI items. These improvements should appear in accessibility options as soon as a device is updated to iOS 13 or Catalina.

Gadgets – TechCrunch

Google improves Android App Bundles and makes building Instant Apps easier

Google is launching a number of new features for Android app developers today that will make it easier for them to build smaller apps that download faster and to release instant apps that allow potential users to trial a new app without having to install it.

Android App Bundles, a feature that allows developers to modularize their apps and deliver features on demand, aren’t a new feature. The company announced these a while ago and there are now “thousands of app bundles” in production and that the average file size reduction is 35 percent. With today’s update, Google is making some changes to how app bundles handle uncompressed native libraries that are already on a device. Those will lead to downloads that are on average 8 percent smaller than before and take up 16 percent less space on a device.

Talking about size, Google now lets developers upload app bundles with installed APK sizes of up to 500 megabytes, though this is currently still in early access.

In addition, App Bundles are now supported in Android Studio 3.2 stable and Unity 2018.3 beta.

While small app sizes are nice, another feature Google is announcing today will likely have a larger impact on developers and users alike. That’s because the company is making some changes to Instant Apps, a feature that allows developers to ship a small part of their apps as a trial or to show a part of the app experience when users come in from search results — and there’s no need to download the full app and go through the (slow) install procedure.

With this update, Google is now using App Bundles to let developers build their instant apps. That means they don’t have to publish both an instant app and an installable app anymore. Instead, they can enable their App Bundles to include an instant app and publish a single app to the store. Thanks to that, there’s also no additional code to maintain.

Developers can now also build instant apps for their premium titles and publish them for their pre-registration campaigns, something that wasn’t previously an option.

Other updates for Android developers include improved crash reports that now combine real-world data from users with that from the Firebase Test Lab when Google sees those crashes under both circumstances. There are also updates to how developers can set up subscription billing for their apps and a couple of other minor changes that you can read about here.

Android – TechCrunch

PlayStation 4’s new beta software reduces clutter, improves multitasking

20th Anniversary PS4, DualShock 4 and PlayStation 4 Camera The beta for PlayStation 4’s 4.00 operating system arrives tomorrow for those enrolled, and Sony wants to give everyone a preview of what it brings to the table. The most noteworthy features are improved library organization, as well as UI tweaks that mean the Share and Quick Menus don’t drop you out of the gaming action.
Folders are new in PS4 v 4.00, letting you group together… Read More

Gadgets – TechCrunch