Chapter 9 Get on the Road!
Use two terminals for raspberry
first teminal run jgp streamer
cd /home/pi/Sunfounder_Smart_Video_Car_Kit_for_RaspberryPi/mjpg-streamer/mjpg-streamer
sudo sh start.shsecond terminal run tcp_server.py:
cd ~/Sunfounder_Smart_Video_Car_Kit_for_RaspberryPi/server
sudo python tcp_server.py
on mac terminal in environment sunfounderPy27
Type in the following address at the address bar of your browser (Firefox is recommended):
http://192.168.178.67:8080/stream.htmlsource activate sunfounderPy27or if openCV is needed
source activate carnd-term1 9.1 install carnd-term1
git clone https://github.com/udacity/CarND-Term1-Starter-Kit.git
cd CarND-Term1-Starter-Kit
conda env create -f environment.yml
conda install -c anaconda tk
cd /Users/uwesterr/CloudProjectsUnderWork/ProjectsUnderWork/RoboCar/sunfounder/Sunfounder_Smart_Video_Car_Kit_for_RaspberryPi/client
sudo python client_App.py9.2 Install Xbox 360 controller
install Xbox 360 controller drive on mac https://github.com/360Controller/360Controller/releases
NOTE!!!
it seems necessary to have the controller plugged into the USB port during boot up
According to http://www.philipzucker.com/python-xbox-controller-mac/ Install pygame
python -m pip install -U pygame --useror
brew upgrade sdl sdl_image sdl_mixer sdl_ttf portmidi
python3.6 -m venv anenv
. ./anenv/bin/activate
pip install https://github.com/pygame/pygame/archive/master.zipcreate a jupyter notebook “xboxControllerOnMac.ipynb”" with
import pygame
import sys
import time
import socket
import cPickle as pickle
UDP_IP = "127.0.0.1"
UDP_PORT = 5005
MESSAGE = "Hello, World!"
print "UDP target IP:", UDP_IP
print "UDP target port:", UDP_PORT
print "message:", MESSAGE
sock = socket.socket(socket.AF_INET, # Internet
socket.SOCK_DGRAM) # UDP
pygame.init()
pygame.joystick.init()
clock = pygame.time.Clock()
print pygame.joystick.get_count()
_joystick = pygame.joystick.Joystick(0)
_joystick.init()gives:
UDP target IP: 127.0.0.1
UDP target port: 5005
message: Hello, World!
1
watch out for the “1” which indicates that the controller was identified
while 1:
for event in pygame.event.get():
if event.type == pygame.JOYBUTTONDOWN:
print("Joystick button pressed.")
print event
if event.type == pygame.JOYAXISMOTION:
#print _joystick.get_axis(0)
#print event
if event.axis == 0: # this is the x axis
print event.value
if event.axis == 5: # right trigger
print event.value
xdir = _joystick.get_axis(0)
rtrigger = _joystick.get_axis(5)
#deadzone
if abs(xdir) < 0.2:
xdir = 0.0
if rtrigger < -0.9:
rtrigger = -1.0
MESSAGE = pickle.dumps([xdir,rtrigger])
sock.sendto(MESSAGE, (UDP_IP, UDP_PORT))
clock.tick(30)when using controller the following output was created
0.00781273842586
-1.00003051851
Joystick button pressed.
<Event(10-JoyButtonDown {‘joy’: 0, ‘button’: 6})>
0.0
0.00781273842586
Joystick button pressed.
<Event(10-JoyButtonDown {‘joy’: 0, ‘button’: 12})>
0.0312509537034
9.3 Implement Xbox controller as input in client_App.py
In client_App.py the part for driving forward extracted
from Tkinter import *
from socket import * # Import necessary modules
ctrl_cmd = ['forward', 'backward', 'left', 'right', 'stop', 'read cpu_temp', 'home', 'distance', 'x+', 'x-', 'y+', 'y-', 'xy_home']
top = Tk() # Create a top window
top.title('Sunfounder Raspberry Pi Smart Video Car')
HOST = '192.168.178.67' # Server(Raspberry Pi) IP address
PORT = 21567
BUFSIZ = 1024 # buffer size
ADDR = (HOST, PORT)
tcpCliSock = socket(AF_INET, SOCK_STREAM) # Create a socket
tcpCliSock.connect(ADDR) # Connect with the server
# =============================================================================
# The function is to send the command forward to the server, so as to make the
# car move forward.
# =============================================================================
def forward_fun(event):
print 'forward'
tcpCliSock.send('forward')then keystrokes are binded to the forward function, this needs to be changed to bind Xbox controller values to the function
# =============================================================================
# Bind buttons on the keyboard with the corresponding callback function to
# control the car remotely with the keyboard.
# =============================================================================
top.bind('<KeyPress-w>', forward_fun) # Press down key 'w' on the keyboard and the car will drive forward.from https://github.com/martinohanlon/XboxController/blob/master/XboxController.py
JOYAXISMOTION
event.axis event.value
0 - x axis left thumb (+1 is right, -1 is left)
1 - y axis left thumb (+1 is down, -1 is up)
2 - x axis right thumb (+1 is right, -1 is left)
3 - y axis right thumb (+1 is down, -1 is up)
4 - right trigger
5 - left trigger
JOYBUTTONDOWN | JOYBUTTONUP
event.button
A = 0
B = 1
X = 2
Y = 3
LB = 4
RB = 5
BACK = 6
START = 7
XBOX = 8
LEFTTHUMB = 9
RIGHTTHUMB = 10
9.3.1 Make steering proportional to remote control lever position
change code from
if event.axis == 0: # this is the x axis
if event.value > thresSteerHigh:
tcpCliSock.send('right')
if event.value < thresSteerLow:
tcpCliSock.send('left') to
if event.axis == 0: # this is the x axis
if event.value > thresSteerHigh:
tcpCliSock.send('right')
angle = int(100*abs(event.value))
data = tmp1 + str(angle)
print 'sendData = %s' % data
tcpCliSock.send(data) # Send the speed data to the server(Raspberry Pi)
if event.value < thresSteerLow:
tcpCliSock.send('left')
angle = int(-100*abs(event.value))
data = tmp1 + str(angle)
print 'sendData = %s' % data
tcpCliSock.send(data) # Send the speed data to the server(Raspberry Pi)9.4 Get IP adress of raspi in shack
Check in the router for the IP adress, procedure is dependent on router At shackspace go to http://leases.shack/#/ (only available from the shackspace network)
and then connect via ssh pi@ipAdress
if you get
Uwes-MBP:data uwesterr$ ssh pi@10.42.26.33
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
SHA256:aYMRAzv3GpxqJNugz1oTi20m0QKVIVfVxszQkJNbNqg.
Please contact your system administrator.
Add correct host key in /Users/uwesterr/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /Users/uwesterr/.ssh/known_hosts:20
ECDSA host key for 10.42.26.33 has changed and you have requested strict checking.
Host key verification failed.then you have to remove the cached key for donkeypi-uwe (or the old IP adress) on the local machine:
ssh-keygen -R donkeypi-uwe9.5 Store snapshots of video stream on local computer
Donkey car stores training data on raspi SD card. In this concept the XBox controller is connected via USB to a laptop (in my case a Mac) and it makes sense to store the traing data on the laptop as well since we anyway will train the NN on that machine.
The URL of the stream is:
url = “http://10.42.26.33:8080/?action=stream”
for a snapshot the URL is url = “http://10.42.26.33:8080/?action=snapshot”
check video https://youtu.be/2xcUzXataIk?t=556 for a good explanation of how to receive an IP video stream. Based on that tutorial the follwing code now stores a single frame and shows that frame as well.
import cv2
import numpy as np
import urllib
# based on example in https://www.youtube.com/watch?v=2xcUzXataIk
url = "http://192.168.178.67:8080/?action=snapshot"
imgNp = np.array(bytearray(imgResp.read()), dtype = np.uint8)
img = cv2.imdecode(imgNp,-1)
cv2.imshow("test",img)
cv2.imwrite( "Snapshot.jpg", img );
cv2.waitKey(10000)9.5.1 Jie Hou’s alternative
Jie has another solution for the same task, see https://drive.google.com/drive/folders/10U8ZTr_2HVnBWrFqvVFGO0UyQn0vCmiE
The code is
import cv2
import numpy as np
try:
from urllib.request import urlopen
except ImportError:
from urllib2 import urlopen
print('# capture image from video #')
stream = urlopen('http://192.168.0.101:8080/?action=stream')
bytes = bytes()
FlagSaveImage = 0
while True:
bytes += stream.read(1024)
a = bytes.find(b'\xff\xd8')
b = bytes.find(b'\xff\xd9')
print(' #a: ', a, ' ,b: ', b)
if a != -1 and b != -1:
jpg = bytes[a: b + 2]
bytes = bytes[b+2:]
image = cv2.imdecode(np.fromstring(jpg, dtype=np.uint8), cv2.IMREAD_COLOR)
cv2.imshow('image', image)
if FlagSaveImage == 0:
cv2.imwrite('test.jpg', image)
FlagSaveImage = 1
if cv2.waitKey(1) == 27:
exit(0)