Control Systems in Kerbal Space Program

Posted to site: 04/10/18

Drone and quad­copter us­age is at an all time high, but con­trol­ling them is­n’t ex­actly plain sail­ing. In a the­o­ret­i­cal set­ting, ap­ply­ing equal power to each mo­tor should cause the ves­sel to rise straight up. In prac­tice, with in­evitable dis­crep­an­cies in de­sign and man­u­fac­ture, pro­duc­ing such a per­fectly sym­met­ric sys­tem is im­pos­si­ble. So the ques­tion arises, how can we vary the speeds of each mo­tor to sta­bilise and con­trol the air­craft? Such ques­tions are nor­mally an­swered in an en­vi­ron­ment such as MATLAB with Simulink but the video game Kerbal Space Program and ad­don kRPC pro­vide a fun sand­box in which to ex­per­i­ment.

The setup

The goal of this pro­ject was to de­sign, build and at­tempt to con­trol a rudi­men­tary quadcopter” in the video game Kerbal Space Program. To be in with the best chance, the ves­sel should sat­isfy a few de­sign re­quire­ments. Firstly, the craft should be as sym­met­ric and bal­anced as pos­si­ble; more pre­cisely, the cen­tre of mass and cen­tre of thrust should be as close as pos­si­ble so that the craft does­n’t flip over straight away. Secondly, we need the en­gines to be highly re­spon­sive to change in thrust be­cause we will be con­stantly al­ter­ing them to keep the craft fly­ing level. In Kerbal Space Program, this means stick­ing with liq­uid rocket en­gines such as the 48-7S Spark”. Note, we don’t need any en­gine gim­balling be­cause we will steer the craft by vary­ing the thrust across the 4 en­gines.

For the de­sign I will be us­ing, please see Figure 1. This de­sign has one con­sid­er­able draw­back over a con­ven­tional quad­copter, in that there is no way to con­trol the yaw of the craft. This will be dis­cussed more later, but con­trol­ling this axis should­n’t be nec­es­sary to achieve sta­ble flight. Moreover, it should be pos­si­ble to ma­neu­ver the craft in 3D space with­out hav­ing any con­trol of the yaw axis (although it will be point­ing in a ran­dom di­rec­tion).

Quadcopter front view
Quadcopter top view
Figure 1 : Jeb’s rocket-pow­ered quad­copter

Next, we need some way of mea­sur­ing the cur­rent state of the ves­sel, pro­cess­ing that in­for­ma­tion some­how, and then ad­just­ing the thrust on each en­gine ac­cord­ingly. There are two com­pet­ing meth­ods for achiev­ing this goal, namely the two mods kRPC and kOS. kOS is ar­guably sim­pler to im­ple­ment but all scripts must be writ­ten in its cus­tom pro­gram­ming lan­guage. Conversely, kRPC al­lows you to write your scripts in C++, Java, Python or many other lan­guages. In this pro­ject, I will be us­ing kRPC with Python.

Model of Quadcopter
Figure 2 : Basic model of quad­copter

Figure 2 de­fines the ba­sic model of the quad­copter that we will be fly­ing, in­clud­ing the num­ber­ing of the en­gines 0 through 3 and the de­f­i­n­i­tions of pitch, roll and yaw.

Control sys­tems

The ba­sic prob­lem pre­sented is how to con­trol the thrust of each en­gine \( T_i \) over time in or­der to achieve some de­sired state (normally re­ferred to as the set state). In this case, for sim­plic­ity, we aim for a set state with both zero pitch and roll, and al­ti­tude at a fixed height. In a noisy sys­tem find­ing an an­a­lytic so­lu­tion to this prob­lem is im­pos­si­ble, there are sim­ply too many vari­ables which can­not be ac­cu­rately mea­sured or pre­dicted. The so­lu­tion must there­fore be re­ac­tive; the sys­tem will have to be con­stantly mea­sur­ing the state of the ves­sel and mak­ing ad­just­ments ac­cord­ingly. As shown in Figure 3, we must de­sign a con­troller which ac­cepts our set state and the cur­rent state and ad­justs the en­gines ac­cord­ingly.

Feedback loop diagram
Figure 3 : Feedback loop for quad­copter con­trol sys­tem

How do we de­sign such a con­troller? A first idea would be a pro­por­tional con­troller; the cur­rent er­ror in the sys­tem is mul­ti­plied by some pre­de­fined con­stant (or gain) and the out­put is the con­trol sig­nal sent to the en­gines. Denoting the cur­rent er­ror as \( e(t) = x_* - x(t) \) where \(x_*\) is the de­sired set state, we can ex­press the con­trol sig­nal out­putted to the en­gines as \[c(t) := k_p e(t)\] for some given gain \( k_p \). To test out this idea, I did a quick test in MATLAB. To sim­plify the prob­lem, we re­duce to a 1 di­men­sional sys­tem; a point which starts at \( x = 5 \) and is aim­ing for state \( x = 0 \). At each time step the state is up­dated and then the ac­cel­er­a­tion of the point is set to the con­trol sig­nal of the pro­por­tional con­troller plus an air re­sis­tance” term. Click here to down­load the m-file I used to model this con­trol sys­tem.

Proportional Control Model
Figure 4 : Modelling a pro­por­tional con­troller

Figure 4 shows how this sys­tem evolves over time; there is a clear is­sue. The con­troller is only aware of the cur­rent er­ror in the sys­tem so it keeps wildly over­shoot­ing the set state and the point ends up os­cil­lat­ing around the set state. The re­duc­tion of the am­pli­tude of these os­cil­la­tions is due only to the damp­ing in the sys­tem from air re­sis­tance. After finely tun­ing the gain, a pro­por­tional con­troller might be ac­cept­able in some ap­pli­ca­tions, but it would make for a very nau­se­at­ing flight for Jeb and may even shake the craft apart.

PID con­trol

A PID con­trol sys­tem aims to rem­edy the is­sue with the pro­por­tional con­troller demon­strated above. The name of this con­trol sys­tem rep­re­sents its con­stituent parts; the pro­por­tional, in­te­gral and de­riv­a­tive terms. The PID con­trol sys­tem can be ex­pressed math­e­mat­i­cally as

\[c(t) := k_p,e(t) + k_i \int_0^t e(x);dx + k_d \frac{de(t)}{dt} \]

where \(e(t)\) is the er­ror at time \(t\), \(c(t)\) is the out­put con­trol sig­nal at time \(t\), and \(k_p\), \(k_i\) and \(k_d\) are the pro­por­tional, in­te­gral and de­riv­a­tive gains re­spec­tively. Note, the pro­por­tional term of this sys­tem is iden­ti­cal to be­fore, this term deals with the cur­rent er­ror in the sys­tem. The next term is the in­te­gral term. Intuitively, this term adds up all pre­vi­ous er­rors in the sys­tem and so rep­re­sents past er­ror. The fi­nal term is the de­riv­a­tive term which cal­cu­lates how quickly the er­ror is chang­ing. This term is very im­por­tant be­cause it is the only term which pre­dicts the fu­ture. It can be viewed as the brak­ing term be­cause it will re­duce the ac­cel­er­a­tion as the sys­tem ap­proaches it de­sired state.

We can model this con­troller in a sim­i­lar way to the pro­por­tional con­troller, to see how they com­pare. Viewing the sim­u­la­tion in Figure 5, we clearly see that the PID con­troller is do­ing a much bet­ter job. The sys­tem has set­tled down into the set state in un­der 2 os­cil­la­tions, and be­fore time \(t = 4 \times 10^4\), whereas the pro­por­tional con­troller was still os­cil­lat­ing around the set state with a large am­pli­tude at time \(t = 5 \times 10^5 \). Click here to down­load the m-file I used to model this con­trol sys­tem.

PID Control Model
Figure 5 : Modelling a PID con­troller

PID con­trollers are fea­tured in many mod­ern drones across nu­mer­ous ap­pli­ca­tions and thus it was the nat­ural choice. For this pro­ject we will only need 3 PID con­trollers, one for pitch, one for roll and one for al­ti­tude, but real drones would also in­clude a PID con­troller for yaw. The next ques­tion to an­swer is how these con­trol sig­nals should be com­bined to con­trol the en­gines. Note if we wish to in­crease the al­ti­tude of our craft we should in­crease the power of all en­gines. To in­crease the roll of our craft we should in­crease \(T_0\) and \(T_2\) while de­creas­ing \(T_1\) and \(T_3\). Finally, to in­crease the pitch of our craft we should in­crease \(T_0\) and \(T_1\) while de­creas­ing \(T_2\) and \(T_3\). This gives us the fol­low­ing pow­ers for each of the en­gines:

\[ \begin{align} T_0 &= -c_{pitch}(t) - c_{roll}(t) - c_{alt}(t),\\T_1 &= -c_{pitch}(t) + c_{roll}(t) - c_{alt}(t),\\T_2 &= +c_{pitch}(t) - c_{roll}(t) - c_{alt}(t),\\T_3 &= +c_{pitch}(t) + c_{roll}(t) - c_{alt}(t). \end{align} \]

The fi­nal prob­lem is how to tune the gains \(k_p\), \(k_i\) and \(k_d\) for each of the con­trollers. In the world of quad­copters the PID tun­ing of your craft is a very per­sonal choice and re­flects how re­spon­sive or smooth you want the ves­sel to be. There is not nec­es­sar­ily a cor­rect tun­ing for a given craft let alone across all craft. There are plenty of good re­sources on­line that guide you through the process of tun­ing these gains. Likewise, there are plenty of com­pet­ing al­go­rithms (including us­ing a ge­netic al­go­rithm) for au­to­mat­i­cally tun­ing PID con­trollers with var­i­ous dif­fer­ent goals in mind.

Working ex­am­ple

This sys­tem has one prob­lem which be­came ap­par­ent fairly quickly in test­ing. The prob­lem arises if you re­lease the drone at a great height (say 5km) and ask it to achieve a low al­ti­tude (say 100m). Now, the dif­fer­ence be­tween the drone’s tar­get al­ti­tude and ac­tual al­ti­tude is mas­sive so \(c_{alt}(t)\) be­comes very large and neg­a­tive. This has the ef­fect of drown­ing out the pitch and roll con­trol sig­nals so the en­gines will not fire at all. Consequently, the drone will tum­ble out of the sky out of con­trol which may be im­pos­si­ble to re­cover from, re­sult­ing in cat­a­strophic fail­ure. The so­lu­tion was to clamp \(c_{alt}(t)\) be­tween -0.8 and 0.8 to keep the sig­nal from be­com­ing over­whelm­ing.

If you would like to down­load the Python file which con­trols the quad­copter in Kerbal Space Program please click here. To use this script you will need to have Python in­stalled lo­cally and the kRPC mod in­stalled in your in­stance of KSP. Also, so that the script can prop­erly de­tect the en­gines, you will need to set an ini­tial thrust lim­iter on the en­gines where en­gine \(T_i\) has a thrust lim­iter of \(i/10\).

Figure 6 : Jeb’s quad­copter in ac­tion

Figure 6 shows a demon­stra­tion of the PID loop. First Jeb gets launched up­wards with all en­gines fir­ing equally, and nat­ural asym­me­try in the craft causes the quad­copter to start pitch­ing down. Then we ac­ti­vate the PID con­trollers and the flight lev­els out, set­tling at an al­ti­tude of 100m. By set­ting the tar­get al­ti­tude to ground level (approximately 35m here) we can land Jeb safely back on the ground. Throughout this video you can see the thrust lim­iters of each en­gine be­ing au­to­mat­i­cally ad­justed by the con­trol sys­tem to keep the craft at the set state. Note I have set the de­riv­a­tive gain on the al­ti­tude loop rather high (2.0) be­cause over­shoot­ing the al­ti­tude on a de­scent could prove fa­tal.

Next steps

So far, all this sys­tem is ca­pa­ble of is main­tain­ing a given pitch, roll and al­ti­tude. It would be pos­si­ble to con­trol these 3 vari­ables man­u­ally to ma­neu­ver the ves­sel but, es­pe­cially with­out yaw con­trol, it would be chal­leng­ing. Ideally, we would like to be able to give the con­troller a point in 3D space which it would nav­i­gate to au­tonomously. This is an­other pro­gram which is dif­fi­cult to solve an­a­lyt­i­cally and so could po­ten­tially solved by an­other PID loop. This new sys­tem would cal­cu­late the vec­tor to the tar­get po­si­tion and feed this into a PID loop, pro­duc­ing the new set state for the pitch, roll and al­ti­tude which would then be achieved by an­other PID loop. Such a sys­tem is called cas­cade con­trol. Moreover, as the con­trol sys­tem is based on a feed­back loop, the tar­get point need not be sta­tion­ary so the drone could be pro­grammed to chase a tar­get ve­hi­cle.

In Kerbal Space Program, a PID con­trol sys­tem could be used to ex­e­cute a pow­ered de­scent in or­der to safely bring a rocket first stage back to the ground, sim­i­larly to SpaceX’s Falcon 9 se­ries of rock­ets. A cas­cade con­trol sys­tem could also be used to guide the rocket back to the ini­tial launch­pad or an­other safe land­ing site. Although, how to do this in the most fuel-ef­fi­cient way (which is cru­cial for low­er­ing the costs of a reusable launch sys­tem) is more of a ques­tion for op­ti­mal con­trol the­ory.

Another pos­si­ble ex­ten­sion to this pro­ject would be to use the ac­celerom­e­ter in a mo­bile de­vice as an in­put de­vice for the set state so that the quad­copter would mimic the ori­en­ta­tion of the de­vice. kRPC opens a server for in­ter­act­ing with the game which can be ac­ces­si­ble on the en­tire net­work, so this con­trol sys­tem could run en­tirely as a self-con­tained ap­pli­ca­tion on the mo­bile de­vice. Although, this may in­cur la­tency is­sues de­pend­ing on the speed of the net­work.

Another di­rec­tion for this pro­ject would be to ex­plore dif­fer­ent con­trol sys­tems. One such sys­tem to ex­plore could be the lin­ear-qua­dratic reg­u­la­tor (LQR). An LQR seeks to min­imise a qua­dratic cost func­tion sub­ject to a sys­tem of lin­ear dif­fer­en­tial equa­tions. The dif­fi­culty in de­sign­ing such a sys­tem is prop­erly defin­ing the cost func­tion and is usu­ally an it­er­a­tive process, sim­i­lar to tun­ing a PID con­troller.

In Kerbal Space Program it’s prob­a­bly safe to as­sume that the teleme­try read­ings are ac­cu­rate but in the real world we are not blessed with such re­li­able sen­sors. A Kalman fil­ter takes the read­ings from the sen­sors and com­bines that in­for­ma­tion with a model for the craft to pro­vide a best guess” as to the ac­tual ori­en­ta­tion and po­si­tion of the craft. This al­lows the con­trol sys­tem to fil­ter out noise from un­re­li­able sen­sors and pro­vide a much more sta­ble flight which is­n’t as re­ac­tive to ran­dom noise in the sys­tem.