BEYOND and the X-Box Kinect 360

Note that this function is deprecated

Pangolin’s BEYOND Software is compatible with the X-Box Kinect 360 Controller, and this combination allows you to create a wide variety of interactive laser effects.

Before using the X-Box Kinect together with the BEYOND software, please install all software relating to the Kinect, and ensure that it is functioning properly. The Kinect controller can be used together with the Kinect SDK and toolkit.

Once the Kinect is installed, connected, and functioning properly, start the BEYOND software.

If BEYOND detects that the Kinect software has been correctly installed, then a button will appear on the toolbar (it looks like a green line). Click on this button, to call up the Kinect window.

The Kinect window in BEYOND operates together with the X-Box Kinect, and supplies data into other parts of the BEYOND Software. There are two types of data that can be used inside of BEYOND. The first is raster data, and this can be used for real-time tracing. The Kinect uses an IR camera, so that it can work in dark conditions. The second type of data that BEYOND accepts is information about the “skeleton” i.e. the position of the main parts of the human body.

There are 20 points in the skeleton. Each point has X, Y, Z coordinates and an activity flag. If the point is not visible (meaning it can not be recognized by the Kinect) than the value of the active property is zero. If the point is visible, than the value of the active property is one.

1. As a real-time raster tracer.
2. Using PangoScript, you can setup two objects: skeleton 1, and skeleton 2. With each of these objects, you can also access all 20 points, within the specified skeleton.
3. Effects, LD2000-style abstract frames, and all places that use the concept of “input”. This area of the software uses a similar logic as is used for FFT, MIDI, DMX, GamePad, or Channel data.

Shape in FIFO (first in, first out) mode. Shape will allow you to record the change of any inputs, further allowing you to record a change in Kinect data, creating a sort of “paint in the air” type laser effect.

1. Synthesized Image (SI for short). SI is a well known container for effects and images. Each node of SI has it’s own settings, and one of them is a setting specifically designed for the Kinect controller.
2. Universe. This is mostly for fun. With the BEYOND Universe, we tried to see how the Kinect could be used for control of components throughout the Universe portion of the software, such as the Slider, Button, XY Pad, and Wheel. As a first step, the results look very interesting.

When you want to use the Tracer Function within this setup, click on the “Stream to” and “Tracer” buttons. Once the Kinect recognizes the body, it will appear as a white body contour. NOTE: It is very helpful to separate the body contour and the background. If you do need to see the background, increase the value of the “Background Spinbox”. For tracing, it is recommended to keep this in a “0” state.

Right now, the Tracer provides a basic set of controls. The Tracer was implemented as an editor and has a “Show it Now” button, that is widely used across other Pangolin software as well.

On the screenshot above, you will see a trace of the raster, that includes the background. We do not recommend using this feature, in this way. To help get a cleaner output, keep the background black as shown in the screenshot below.

After you click on the Kinect button on the status bar, you can minimize the Kinect window, but be sure not to close this - only minimize it. Kinect’s skeleton coordinates will be visible in the preview panel, and you can see that data as it comes from the Kinect into BEYOND.

Again, BEYOND supports two skeletons, so you may see a second skeleton as soon as BEYOND detects it (i.e. if another person were to step into the Kinect’s view).

Using Effects with the Kinect is simple. Take Oscillating Effects for example. By default, it works depending on time. In addition to time, you can use the twenty points of the skeleton. Click on “Input”, select Kinect, and choose the points you want. If you want to use the position, then you will need to add two effects - Position X, and position Y.

The advantage of this method, is that it can be used in all types of effects, and all other places where “Input” is used.

SI offers three basic actions. We have plans to add many more as well, in upcoming builds of BEYOND.

Action 1 - Position: To enable this, select the point’s name in the “Simple p\Position” dropdown box. Synth will use X and Y coordinates. Amplification will allow you to increase the amplitude. Open “Hide Inactive” - And select “yes”, and it will hide the Synth item, if the point is not recognized.

Action 2 - Fit into rectangle: This works well with your hands. The idea is to fit the item into the rectangle (XY) between the hands. Your hands will diagonally define the corners of the rectangle.

NOTE: Do not use this Action, together with Action 1

Action 3 - Only make this visible, if the “Z” coordinate is in the specified range: Here you define the minimum and maximum value of the “Z” coordinate. The idea is that you stay behind the “Z” coordinate range, and move your hand forward into the range. It makes a Synth item visible. When you take your hand back, it makes the image invisible. There may be variations with body

An interesting array of effects can be created, by means of Shape in FIFO mode. The Shape is able to record values of inputs, and use it as a base frame. Inside of the top most panel, specify what inputs you want to use. NOTE: that you may create a couple of Shapes and put those into the Synthesized Image.

Parameters that will influence the results are: number of points (first tab), Recorder panel, Physics, and of course Scrolling. In fact, the same set of logic is applicable to an FFT recording. We would also like to note, that you can mix different kinds of Inputs directly or indirectly. For example, if you want to have a ball jumping up and down that follows your hand, you can use the Kinect for base motion, and another Y position for an FFT effect Input.

Also, a reminder about the Shape Oscillators. There are Amplitude, Waves, Speed, and TimeShift sliders. The Speed slider and Input may not work well at this time, as we are improving this area, but the Amplitude and other sliders should work just fine. We encourage you to play with these features.

This function works in such a way, that BEYOND will detect the distance of the “Z” coordinates of the shoulder and hand. If you move your hand in front of you, then you should see a red circle. This means that your hand is “active” and in a ready state. In this state, the X and Y coordinates go into component, the same as the mouse coordinates would. In general, our idea here was to allow clients the ability to touch and control the sliders, knobs, and buttons they’ve created inside of the BEYOND Universe.

As a test, there is an ability to touch the pads, sliders, zone preview panels, wheel, etc. You can even see the visualized skeleton points in the background, and you will notice that your hand position is indicated in red. We encourage you to try and experiment with these features as well, and we also readily welcome feedback in this area.

Does every model of Kinect work?

You need the Kinect that came with the X-box 360. Newer kinects do not work. To use Kinect the following requirements need to be met:

• Kinect requires BEYOND Ultimate 32 bit to operate.
• Kinect dev kit version 1.8.
• Kinect 360 USB / kit for Windows

Can I install the Microsoft Kinect SDK on windows 10?

If you want to install the dev kit 1.8 on windows 10, run the program in compatible mode and check “windows 8.1” and follow the instructions below.

1. Right click on setup file.
2. Click on Properties.
3. Click on Compatibility tab.
4. Put a check mark on Run this program in compatibility mode for.
5. Select Windows 8.1.
6. Put a check mark on Run this program as an administrator under Privilege Level.

Where can i find the 32 bit version of BEYOND?

Contact support@pangolin.com