Using accelerometer and EMG signals to estimate arm motion - Dissertation Example

?The purpose of this study is to understand the feasibility of predicting arm movement trajectories based on features that are extracted from EMG signals, as a result of muscular action, and accelerometer input. as a result of acceleration of limbs to movement. These features were measured through a set of muscles that were assumed to be present in transhumeral amputee patients. It is known that myoelectric prosthesis suffer from pattern recognition degradation in real settings (Fougner, et al., 2011). This study investigates a means to overcome this degradation through use of EMG signals combined with accelerometer signals to measure the upper arm static and dynamic acceleration. Both EMG signal and accelerometer inputs are fed into an artificial neural network. The artificial neural network continuously predicts arm movement trajectories. An offline time-delay Artificial Neural network (TDANN) is employed to predict the movement trajectories of the arm. The accuracy of prediction was judged by using a set of goniometer readings which provides the changes in the angles of the upper limb. All data was processed in the Matlab environment. The TDANN deployed was developed in the neural network toolbox present within the Matlab environment. The developed neural network was optimized and trained with different sets of inputs, and the results for each of the trails was noted. The results obtained clearly demonstrated that accelerometers are able to enhance pattern recognition and thus provide better prosthesis control. Neural Network Optimization and Prediction Performance The neural network structure used for the study is the TDANN. TDANN is a neural network architecture whose primary purpose is to function on continuous data. The major advantage of using TDANN on continuous data is its ability to adapt the network’s weights and activation function online by use of back propagation error method (Fougner, et al., 2011). The networks can be visualized as a feed forward neural network which is trained for time series prediction. The architecture has continuous inputs that are delayed and sent into the network. In this study, the inputs to this neural network architecture were delayed time series; that is the previous values of 10 channels for 4 for EMG and 6 channels for accelerometers. The measured goniometer signals served as desired output of the TDANN and also as the present state of the time series. The usage of one- layer time delay artificial neural network which is a feed forward structure allow us to predict continuous trajectories which is advantageous for a coordinated and simultaneous control of multiple degrees of freedom in a natural manner. The use of delayed input signals enabled the neural network to capture dynamic input-output properties and account for the delay between the onset of the muscle activity and mechanical arm movement (the activation of the hardware motors in the prosthesis) (Fougner, et al., 2011). TDANN have also an advantage of rapid training time when compared to the dynamic neural networks with recurrent connections. We investigated using a TDANN to predict the elbow flexion degrees, wrist flexion degrees, wrist deviation degrees and forearm rotation degrees based on EMG information from the available intact muscles in transhumeral amputation patients. The EMG information was combined with the accelerometer information about the upper arm and the upper trunk orientations. A one layer time-delay artificial neural network (TDANN) was created using Matlab’s neural network toolbox; this network was used to capture the time-series data (EMG and accelerometer signals as an input with the goniometers and torsiometer signals as output). The size of the hidden layer was set by default to be 10 neurons and the network was trained then the hidden layer size was increased to 25 then to 35 and the performance of the network was monitored. TDANN with 35 neuron hidden layer size was then chosen. The network used 2 input delays to allow building a dynamic network, which has memory so the output will depend not only on the current inputs, but past values of the input. By taking two input delays two past inputs were included in the training process and because the data was resampled at 30 Hz, every past point is 1/30 seconds in the past of the current point; the current output is dependent on the past 2/30 seconds of the inputs. Increasing the number of input delays caused memory problems since more input delays will increase the number of weights and connections in the network, so the network was kept on two delays. The initial weights and biases were assigned randomly. Then the input data was divided to form the training, validation and testing sets in a ratio of 70%, 15% and 15% respectively. The network was trained in different ways, first by increasing the number of hidden layer size secondly by changing the input sets to see the effect of each input on the training performance. Initially the neural network was trained with a default hidden layer consisting of 10 hidden neurons. The number of hidden layers was then increased to 25 and 35 hidden neurons. A further increase in the number of hidden neurons was limited due to limited memory in Matlab. RMSE (Root mean square errors) and correlation coefficients (R) were calculated for each training trial, once the best training was achieved the errors were manipulated by normalizing them over the full range of movement for the wrist deviation, flexion and the elbow flexion and the forearm rotation for each subject separately. The best training network was then used to simulate the network for the movement one at a time for the first subject, and the RMSE and R were calculated. The RMSE values were divided into RMSE1, RMSE2, RMSE3 and RMSE4. The R values were demarcated in the same manner i.e. R1, R2, R3 and R4. The numerical values in each of the values denoted a specific sort of motion; wrist deviation is denoted by 1, wrist flexion by 2, elbow flexion by 3 and forearm rotation by 4. The results showed that the RMSE values for all four motions decreased with an increase in the number of hidden neurons. This signified that with an increase in the number of hidden layers the prediction accuracy for the neural network also increased. The R values for all four motions increased as the number of neurons in the hidden layer increased. This pushes to the conclusion that a strong correlation was present. The Results and Effects of Removing Inputs on Training Performance The study consisted of a total of four inputs; two sets of EMG signals and two sets of accelerometer signals, ACCF1 and ACCF2. In order to understand the affect of removing a set of input the following trails were conducted; EMG signals plus ACCF1 plus ACCF2, EMG signals plus ACCF1, EMG signals plus ACCF2 and EMG signals only. The trail was conducted on the same nine subjects with a constant sampling rate of 30Hz, hidden layer of 35 neurons and a whole set of movements. The RMSE and R values were used to quantify the results obtained. In the case of EMG signals with all accelerometer inputs the values are shown in the table below; Subject’s ID RMSE1 RMSE2 RMSE3 RMSE4 R1 R2 R3 R4 DB 5.49 7.35 8.03 7.57 0.840 0.876 0.973 0.940 AA 4.90 9.06 9.92 8.64 0.888 0.904 0.956 0.930 AD 7.99 8.55 8.97 6.38 0.876 0.844 0.971 0.949 NN 6.90 8.11 7.95 7.25 0.853 0.898 0.981 0.904 MH 5.46 9.37 14.95 7.27 0.683 0.809 0.923 0.894 HM 6.83 8.97 11.81 7.97 0.794 0.908 0.952 0.859 CA 7.31 9.67 12.41 7.79 0.820 0.878 0.940 0.859 CM 5.76 8.08 7.42 6.94 0.862 0.889 0.967 0.948 AK 6.27 8.98 8.72 5.68 0.880 0.913 0.973 0.930 MB 7.51 8.65 9.78 7.39 0.852 0.803 0.961 0.901 Mean 6.44° 8.68° 9.99° 7.29° 0.835 0.872 0.960 0.911 STDEV 1.02° 0.69° 2.38° 0.83° 0.060 0.040 0.018 0.034 The values obtained using all the inputs may serve as a baseline with which other combination of inputs may be assessed. In the case of EMG signals and ACFF1 i.e. with all EMG signals and one accelerometer input the following values were obtained. Subject’s ID RMSE1 RMSE2 RMSE3 RMSE4 R1 R2 R3 R4 DB 6.80 9.57 12.62 11.35 0.741 0.773 0.931 0.863 AA 6.80 13.34 14.14 11.63 0.770 0.777 0.909 0.870 AD 9.69 10.62 14.51 8.76 0.811 0.746 0.922 0.901 NN 8.94 11.28 13.06 9.56 0.737 0.792 0.948 0.826 MH 6.27 11.39 18.18 10.65 0.545 0.703 0.883 0.754 HM 7.52 10.53 15.56 9.71 0.741 0.871 0.915 0.781 CA 9.96 14.23 19.22 10.57 0.625 0.709 0.849 0.719 CM 7.85 11.59 12.29 11.14 0.724 0.754 0.906 0.860 AK 7.92 12.28 13.36 7.56 0.797 0.827 0.935 0.871 MB 10.05 10.48 13.83 10.97 0.713 0.691 0.920 0.766 Mean 8.18° 11.53° 14.68° 10.19° 0.720 0.764 0.912 0.821 STDEV 1.40° 1.41° 2.33° 1.29° 0.080 0.057 0.028 0.061 The comparison of the values obtained with all inputs reveals that the mean RMSE value significantly increases. Consider RMSE1 and RMSE3, the RMSE 1 and RMSE3 values for all inputs is 6.44 and 9.99 respectively. With removal of ACFF2 these values increase to 8.18 and 14.68 respectively. This increment signifies the increase in error. Following this, the correlation coefficient, R, also decreases. Considering the same motion, R1 drops 0.115 values and R3 drops 0.048 values. The third case involves EMG signals combined with ACFF2. The RMSE and R values obtained using this combination are given in the table below Subject’s ID RMSE1 RMSE2 RMSE3 RMSE4 R1 R2 R3 R4 DB 6.84 9.01 11.08 9.13 0.739 0.803 0.947 0.913 AA 6.51 12.24 15.60 12.72 0.792 0.817 0.888 0.842 AD 9.33 11.70 17.23 10.30 0.826 0.680 0.889 0.860 NN 9.68 11.58 17.96 9.48 0.685 0.779 0.899 0.830 MH 6.11 12.44 22.51 9.42 0.575 0.626 0.813 0.814 HM 7.33 11.78 19.04 9.69 0.757 0.836 0.870 0.783 CA 9.45 14.72 19.99 10.00 0.673 0.685 0.835 0.753 CM 7.42 11.33 12.67 10.40 0.758 0.767 0.900 0.879 AK 8.02 12.73 14.86 8.62 0.791 0.814 0.918 0.829 MB 9.80 10.98 17.08 10.15 0.730 0.653 0.875 0.804 Mean 8.05° 11.85° 16.80° 9.99° 0.732 0.746 0.883 0.831 STDEV 1.41° 1.44° 3.41° 1.11° 0.073 0.077 0.039 0.046 In comparison to the trial involving all signal inputs, the RMSE values obtained with EMG signals and ACFF2 are higher. The lack of input from ACFF1 has resulted in greater error to be instilled within the output. Comparing RMSE1 and RMSE3, we find that RMSE1 has incremented by a value of 1.61 and RMSE2 by 3.17. The correlation coefficient ‘R’ has decreased. R1 has decreased by a value of 1.03 and R3 by a value of 0.077. The last trial involved use of EMG signals only. The accelerometer inputs were denied to the neural network architecture. The results obtained using a 35 neuron hidden layer are given below Subject’s ID RMSE1 RMSE2 RMSE3 RMSE4 R1 R2 R3 R4 DB 8.95 13.67 22.08 17.07 0.480 0.424 0.769 0.642 AA 9.01 17.68 25.58 17.77 0.534 0.555 0.658 0.657 AD 13.90 13.31 25.70 12.57 0.543 0.550 0.729 0.783 NN 11.90 15.67 29.30 12.72 0.442 0.541 0.702 0.669 MH 6.92 14.11 28.60 13.54 0.380 0.466 0.672 0.550 HM 8.89 14.74 24.99 11.58 0.609 0.736 0.763 0.672 CA 11.35 17.92 27.72 12.14 0.457 0.461 0.646 0.602 CM 9.12 14.41 19.98 16.59 0.599 0.575 0.725 0.650 AK 10.18 17.45 21.92 10.68 0.628 0.601 0.811 0.719 MB 13.33 13.20 21.43 14.79 0.368 0.414 0.796 0.499 Mean 10.35° 15.22° 24.73° 13.95° 0.504 0.532 0.727 0.644 STDEV 2.21° 1.85° 3.25° 2.48° 0.093 0.097 0.058 0.080 The RMSE values obtained using EMG signals only is quite higher when compared to the rest of the trials individually. The R values are also lower than each of the values obtained using the other trials. Comparing RMSE1 and RMSE3 again, we find that there is a difference of 3.91 for RMSE1 and a difference of 14.74 for RMSE3. The R values also differ greatly with R1 having a change of 0.331 and R3 having a change of 0.233. These differences in values signify that the error has increases significantly for architecture having only EMG signals as input, while the correlation decreases as well. Comparison of Prediction Results with Other Studies The most relevant studies conducted with respect to this study have been conducted by Akhtar, et al. (2012) in order to study the feasibility of using shoulder joint angles and upper arm EMG signals to predict distal arm joint angles. The other relevant work was undertaken by Pulliam, et al. (2011), who studied the prediction of dynamic arm movements based on EMG signals acquired from residual upper limb muscles. Akhtar, et al. (2012) studied the feasibility of using a combination of shoulder joint angles and EMG signals harnessed from the biceps, triceps and the deltoid. The signals were given input to a two layer TDANN. A series of five different trials were conducted; shoulder orientation, EMG without forearm channels, EMG with all channels, both shoulder orientation and EMG without forearm channels and lastly both shoulder orientation and EMG with all channels. The results were analyzed through RMSE and R values for each of the listed trials. The results which are comparable to this study are; EMG (all channels) whose values were calculated as RMSE 8.48 and R 0.87 for flexion and extension. In the case of pronation and supination of upper limb the RMSE value was 8.56 and the R value calculated was 0.96. Comparing these values with the study conducted,the RMSE value for flexion and extension was 19.34 and the R value was 0.69. While the RMSE value calculated for supination and pronation was 12.7 and R value was 0.518. In this particular case, it is clear that the EMG electrode position followed by Akhtar, et al. (2012) provided better results. Comparing the RMSE values for shoulder orientation and all electrodes and all EMG channels and accelerometer inputs we find that the RMSE value for this study was 7.64 as compared to 5.57 in the case of flexion and extension of the arm. While in the case of pronation and supination the RMSE values were 9.71 as compared to 5.10. . Comparing the overall degree of success between the two study’s we find that Akhtar et al. (2012) was able to produce more scientifically correlated and accurate results. Pulliam, et al. (2011) studied the feasibility of predicting dynamic arm movements based on the EMG signals acquired from muscles that would be relatively intact in upper limb amputees. The study involved measuring EMG signals and kinematics from seven different muscles during a variety of movements with different complexities. The neural network prediction results for all five subjects with all movements were assessed by calculating RMSE and R values for all movements performed. The mean RMSE and R values for single joint, serial joint and ADL quantitatively summarized the results. These mean RMSE and R values are a means of comparing results obtained within this study. The movements within this study focused on movements that involved ADL. Thus, comparisons would be made with respect to RMSE and R values of ADL. The elbow flexion/extension RMSE and R values deduced by Pulliam, et al. (2011) were 16.7 and 0.84 respectively. Compared with these values, this study obtained RMSE values of 7.9 and an R value of 0.93. The other comparable results involved wrist deviation and flexion. The study conducted Pulliam, et al. (2011) deduced RMSE and R values of 24.1 and 0.68. In comparison this study produced RMSE and R values of 5.0 and 0.77. A closer look at the comparison of RMSE and R values calculated reveal that this study has higher correlation and lower error than the results deduced by Pulliam, et al. (2011). This may lead to the conclusion that the choice of accelerometer inputs and positioning of EMG electrodes has a vast impact on pattern recognition. Muscle Input Selection The group of muscles used for this study were biceps, triceps, pectoralis major and deltoid. Each of these muscle sites is easily accessible through surface electrodes in lab setting. Further, the positioning of the electrodes was strengthened by use of manual muscle tests for each experiment. The EMG signals were detected using surface EMG electrodes rather than intramuscular EMG electrodes. With use of sEMG electrodes, the subject’s were able to perform the given tasks easily, comfortably and without distorting the real motion, which would have been caused with use of intramuscular electrodes. A combination of sEMG electrodes and intramuscular electrodes would not have produced the high end results required (Pulliam, et al., 2011). A feed forward muscle selection has been shown to cause no significant difference between a fixed muscle set i.e. a muscle set based on population statistics and subject-tailored muscle sets i.e. a muscle set which consisted of the biceps, triceps, pectoralis major and/or deltoid. It was proved that there existed no significant change in using all the four muscles or a combination of these muscles (Pulliam, et al., 2011). Four different trails were organized in which a total of three from the four muscles were used to measure RMSE and R values for each of the 10 subjects performing all 4 motions. The four different trials consisted of biceps, triceps and medial deltoid as one group. The second group as biceps, triceps and pectoralis major and the third group consisted of biceps, medial deltoid and pectoralis major. The last group consisted of triceps, medial deltoid and pectoralis major. These trials were conducted with a 35 neuron hidden layer and with both accelerometers providing input. The results from each of the four trials exhibited minimal variation with maximum variation as 0.43 values for RMSE and 0.006 for R values. In conclusion the muscle selection had little effect on the overall outcome of the trials. Control of Terminal Devices Using this Method The transhumeral amputation patients should be provided with a prosthetic terminal device, a claw or hand, which would be an extension to the prosthetic elbow and wrist rotator. The current methods to control a terminal device involve the use of complex body harnesses to control the opening and closing of the terminal device (Akhtar, et al., 2012). A similar control method as described in this study may be implemented to acquire better control of the terminal device. Within transhumeral patients the trapezius muscle can be used to acquire myoelectric control of the terminal device. This would enable patients to have greater control of all joints if the terminal device control is harnessed through EMG signals and accelerometer inputs. Works Cited Akhtar, A., Hargrove, L. J. & Bretl, T., 2012. Prediction of distal arm joint angles from EMG and shoulder. San Diego, IEEE EMBS. Fougner, A., Scheme, E. & AdrianD.C.Chan, 2011. Resolving the Limb Position Effect in Myoelectric Pattern Recognition. IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, 19(6), pp. 644-651. Pulliam, C. L., Lambrecht, J. M. & Kirsch, R. F., 2011. Electromyogram-based neural network control of transhumeral prostheses. Journal of Rehabilitation Research & Development, 48(6), pp. 739-754. Read More