In this article, we’ll be discussing an Output Device, Method for Generating a Machine Learning Model, and a Computer Program by Shimano, US publication 20210009226. The publication date is Jan. 14^th, 2021 and the filing date is July 10^th, 2020. This has not been granted yet.

Brief Summary (tl;dr)

This is a first for me in the bike industry. I haven’t seen anything like this one come across my computer. Shimano are introducing a bike that can sense a ton of different inputs such as cadence, torque, attitude, speed, battery power, etc. and will apply an ‘assist ratio’ to the motor and transmission so the rider can ride the bike longer and faster while using less energy from both the rider and battery. Most importantly, the system can learn you, using machine learning. The bike can understand whether the rider is comfortable or not and can adjust the assist ratio accordingly. In the end, after multiple learning sessions, the bike will be perfectly optimized for one rider to be as comfortable as possible, where the rider and battery use the least amount of energy.

Intro

Wikipedia defines machine learning as the study of computer algorithms that improve automatically through experience and by the use of data. It is seen as a part of artificial intelligence. Machine learning algorithms build a model based on sample data, known as “training data”, in order to make predictions or decisions without being explicitly programmed to do so. Machine learning algorithms are used in a wide variety of applications, such as in medicine, email filtering, and computer vision, where it is difficult or unfeasible to develop conventional algorithms to perform the needed tasks.

Shimano are introducing a machine learning system… for a bike. This system takes sensed inputs from the bike, and sends them to a magic box to perform super calculations, and outputs what they’re calling an ‘assist ratio’. The assist ratio is used to assist in the propulsion of an E-bike at an optimal level. The bike can then, over time, learn how you ride and adjust the assist ratio accordingly. Like any machine learning technique, this bike can actually learn how you ride, and the bike can be optimized based on what it learns about you. Crazy right?

In its simplest form, this system is input with a bunch of sensed information from sensors all over the bike-related to speed, cadence, torque, gear ratios, attitude, and battery power (among others) — battery power is the important part. This information is sent to a machine learning model, which applies weights and biases to calculate the assist ratio to allow the bike to run in an optimal setting, where the rider can ride as fast and as far as possible using the least amount of energy and using the least amount of battery power. The system will control the power output of the pedal assist from the motor and control the gearing for a silky smooth ride. As far as learning, the system will be supervised, meaning the rider will teach the system manually. As the rider continues to teach, the system continues to improve the assist ratio.

The system can even control your brakes, suspension, and seat post, so this can also be for a mountain bike — though these parts are pretty unclear in this document.

Intended Novelty

The explicit intended novelty here appears to be the use of a data acquisition unit that takes power consumption into account when determining optimized riding parameters. According to Shimano, a system already exists that determines an assist ratio using a ton of different input parameters, but none take battery power into account. Shimano want to optimize power consumption and also provide a comfortable and low-stress ride for the rider.

Why

Here’s a pretty concise line on why Shimano are doing this. They want the rider to have a comfortable ride.

The [system] in the present embodiment can control the [bike] so that the electric power consumption efficiency is improved depending on the way of riding of the [bike] as the learning progresses, and allows the user to comfortably travel the [bike] for a long time.

What

I’m going to break this down similarly to the patents I write. We’ll have an architecture section, a machine learning model section, and a training section.

Architecture

Figure 2 shows an overall system diagram that is on-board the bike itself. The bike has sensors (SG) to determine speed, acceleration, angle (pitch/roll/yaw), cadence, and torque. Additionally, there are methods to determine what gear a rider is in, seat post position, and suspension characteristic (stroke/damper/etc.). Lastly, there is an operating device (cycling computer) and sensors on the battery, brakes, and assist mechanism (motor). These sensors output the sensed information to a machine learning model, and the machine learning model outputs an assist ratio to the motor and transmission. Remember, the important parts here are torque, cadence, and battery power.

It’s important to note the directions of the arrows in Fig. 2. The sensors only feed information to the machine learning model (one-way arrows), but components like the assist mechanism, transmission, etc., send and receive information back from the machine learning models (two-way arrows). This means the system receives and sends information to the motor, transmission, etc., to help the bike ride more comfortably.

All of this information is sent through a machine learning model (shown in FIG. 3), which determines an optimal assist ratio. The assist ratio is sent back to the components so the bike can be ridden more comfortably, meaning less stress on the rider and less battery power consumption.

This system is set up to automatically adjust the motor’s power output, change gears (maybe a gearbox), automatically adjust your dropper, automatically adjust your suspension, and automatically apply the brakes based on what the machine learning model determines as optimal. There’s even a GPS unit that can be used to determine your location.

Machine Learning Model

FIG. 3 shows the machine learning model that Shimano have developed. In this model, the input parameters (sensed information) are fed to intermediate layers for calculations, which are then fed to an output layer to determine the assist ratio. This is a fairly generic machine learning model.

More specifically, the intermediate layer is trained to allow a pedaling load to be in a ‘comfort zone’. In short, the system automatically determines the most ideal pedaling efficiency so the rider can go as fast as possible and as far as possible, using the least amount of battery, with the least amount of energy expended by the rider.

The parameters of the intermediate layer include nodes, weights, and biases. All the circles in Fig. 3 are nodes, which mimic the neurons in a brain where certain values are determined. The nodes are linked with biases and weights to the input parameters and pass the calculation to another set of nodes for other calculations. For example, lets take pitch, roll, and yaw and apply weights to the calculation. I would assume that pitch will be weighted much higher into the assist ratio calculation because climbing and descending are much more influential on power consumption compared to roll and yaw. So, roll and yaw should have low biases, while pitch should have a high bias in the final assist ratio. Because this is machine learning, these weights and biases can change as the system learns. There can be as many intermediate layers and nodes as necessary for an acceptable calculation.

The outputs of intermediate layers are then fed to an output layer, which takes the weighted information determined from the intermediate layers to calculate a single assist ratio, which is then applied to the motor.

Shimano state the intermediate layers are trained to allow the rider to maintain in a comfort zone, or a comfortable riding situation.

… the intermediate layer 33 is so trained as to allow the load on the user of the [bike] to fall within a range of comfort when the electric motor Cl is driven in accordance with the information related to an assist ratio output ted from the output layer 32.

In the end, Fig. 5 shows what this entire system tries to do. Your cadence and torque (among other parameters) are determined, and the system can figure out how stressed a rider is and apply the assist ratio to keep you in the high comfort zone, which is noted in the box. The system tries to maintain a comfort level in that box.

If you’ve ever been on a smart trainer, you’ll know a vague idea of what your range is. If you’re pushing the pedals too hard or too soft, you’re out of your comfort zone. If you’re pedaling too slow or too fast, you’re also out of your comfort zone. Remember, this system takes battery power into account so the assist ratio will change as the battery power changes in an attempt to optimize power consumption.

Added based on Jon O’s question below:

It appears as though the initial model will be pretrained prior to a user receiving the bike using training data from other bikes. They also say the training data can come from simulated data.

The machine learning model stored in the storage unit in the initial state may be generated by collecting in advance travel data of the [bike] having high electric power consumption efficiency of the battery as training data and using the training data for supervised learning, though not limited to be generated by simulation.

Training

The last part of this document is the training, which is what makes computers “intelligent”. There are a few types of machine learning training methods, such as supervised, semi-supervised, and unsupervised. Supervised training is manually trained by a user, unsupervised is trained on a dataset, and semi-supervised uses both. Unsupervised is the most efficient but can be very unpredictable, especially with bad/noisy data – the computer effectively trains itself. In the case of this system, Shimano say it’s supervised.

As the training progresses, the system will continually improve to maintain comfort and battery power.

By advancing the training, the processing unit can generate the machine learning model for assisting the traveling of the [bike] by the [motor] as long as possible while the comfortability of the user is maintained.

FIGs 10 and 11 show a little computer that is mounted on the handlebars of the bike. The computer tells the rider what is going on with the bike, such as speed, battery power, gear, etc. There will be a button on the screen which activates the learning mode. In this case, it’s called ‘evaluation of learning’. So, the rider will press this button to activate the learning mode, which takes you to the screen in FIG. 11. If you’re riding comfortably, or if the system makes a change to gearing, for example, you can tell the system if you like or dislike how you’re riding. The system will the learn (adjust) based on your decision. After doing this over and over again, you will teach the system who you are and how you ride. It will then, hopefully, be perfectly set up for a comfortable ride for you and only you.

They even say this training can be done with your voice…

The evaluation input unit in the third example includes a microphone of the voice input-output unit 209 and a voice recognition unit 217. The processing unit 201 recognizes information related to the voice of the user recognized by the voice recognition unit 217, thereby receiving an evaluation.

…or facial expressions. This is hilarious. The system can read your horrible, stressed-out face and can automatically adjust the assist ratio.

The evaluation input unit in the fourth example includes a camera. The communication device 2 includes a camera, for example. The communication device 2 is mounted to the human-powered vehicle A so as to photograph the face of the user by the camera. The processing unit 201 identifies facial expressions of the user from an image acquired by photographing the face of the user with the camera and determines whether the control is proper in accordance with the facial expressions of the user.

Conclusion

Part of my day job it to work on AI patents. I am by no means an expert, but this particular patent is pretty simple compared to what’s actually out there. The difficult part is simplifying it. I fully anticipate more of these in the future, especially when money starts to pour in. I also think we’ll see more of this when laws and infrastructure start to support bikes as a more tangible concept for commuting; rather than a car, bus, or train. But that’s year’s away.

Recently, Pinkbike posted a podcast on important bike inventions, asking what we believe the most important invention is. I wrote a comment about how active bikes (or smartbikes, whatever) will be the most important thing the bike industry will create. I got shit on pretty badly, but I get it. I don’t expect anyone to understand what’s going on here, and how this type of technology will absolutely change what a bike actually is. It’s a bike that can think. Absorb that.

I’d take a guess and say this is probably for a commuter. I’ve always thought commuter bikes should be similar to cars, in that they should be as easy and safe as possible to ride. No one wants to show up to the office drenched in stink and sweat, and this idea may help.

Whether you like it or not, machine learning and AI are running our lives, and it’ll only be more prevalent every day. Now, we can see it could be applied to a bike. We’re getting further into the future with these developments and I’m very interested to see where we’ll be in 10 years. Did anyone think we’d be where we are this quickly? Probably not.