Getting Started on Unity3D Perceptual / Augmented Reality (Step by Step)

After some time playing with the AugmentedFarm I made some annotations and I will pass to everyone, hope that your start will be easyer than mine... I'm not expert and if I say something wrong please let me know.

Let's Go ...


1º - Create a new unity Project

Img - Empty unity project
new project 

2º - Import The Plugin Files from demofolder on the SDK folder... i get them from the from AugmentedFarm:
( C:\Program Files (x86)\Intel\PCSDK\demo\AugmentedFarm\Assets\Plugins )

  1. libpxcupipeline-ot.dll
  2. pxcm-structures-ot.cs
  3. pxcupipeline-ot.cs
  4. tracker.meta
  5. tracker ( Folder )

 obs: the plugins from the others demo are different , they dont have the AR capacity , but this one have the AR and the others

Img - Drag the Plugins Folder
import plugins

3º - Import The Pipeline folder
 C:\Program Files (x86)\Intel\PCSDK\demo\AugmentedFarm\Assets\Pipeline )

  1. SDKPipeline.cs
  2. SDKPipelineObject.prefab ( just a prefab of a empty object with the SDKPipeline.cs code )

Img - Drag the Pipeline Folder
import Pipeline 

4º - Import The Augmented Reality Scripts
C:\Program Files (x86)\Intel\PCSDK\demo\AugmentedFarm\Assets\Book\scripts\AugmentedBook ) 

  1. ABVirtualFarmScript.cs
  2. ABARCameraScript.cs
  3. ABFarmScript.cs
  4. ABStatics.cs

Img - Drag the AugmentedBook Folder and place on Scripts Folder
import AR Scripts

Now we got the basics to start the Augmented Reality...


1º - Reset the Transform from the Camera and Rename it to backgroundARCamera


2º - Drag the SDKPipelineObjectprefab to the Hierarchy on unity Editor, and reset the transform

3º - Create a new GameObject Plane so we can render the background ( RGB Camera ), reset the transform and rotate on X axis -90 and rename it to backgroundARPlane

4º - Create a new GameObject Directional Light and reset the transform for better light , then you can move away the light
5º - Assign the Plane to the SDKPipelineObject on Object To Draw On Variable

We have now our background RGB image rendered if we press Play button

Lets Organize the things a bit ...


obs: dont make modifications while on playmode , because they back to previous when you exit playmode...

1º - Drag the camera, light and plane into the SDKPipelineObject

2º - Scale the plane for : x = 0.4 / y = 1.0 / z = 0.3


3º - Ajust the position of the plane relative to the Camera to get the right position
4º - Select the Camera and Create a new Layer for the Background Image, call it BackgroundAR


5º - Set the Camera Culling Mask to only BackgroundAR


6º - Set the Layer of the Plane to BackgroundAR
7º - Save your scene and Project

Now we can Make the Tracked Object!


0º - Move the SDKPipelineObject to another position

1º - Create a new Camera , call it ARCamera , and reset the transform

2º - Create a new Empty GameObject , call it ARObject, and reset the transform ... you are going to put your models on this gameObject

3º - Create a new Empty GameObject , call it ARVirtualObject, reset the transform and drag ARCamera and ARObject to it

4º - On the ARVirtualObject put the script ABVirtualFarmScript.cs

5º - On the ARCamera put the ABARCameraScript.cs

ps.: the Script look for "AugmentedBookCamera" ... open the script and change to "ARCamera"

6º - On the ARObject put the ABFarmScript.cs

7º - Create a new Layer for the ARObject, call it objectAR

8º - Change Culling Mask for from ARCamera to objectAR

9º - Change the Clear Flags from ARCamera to Depth Only

10º - Change the Depth from ARCamera to 1

10º - Create a new Object Cube and reset the transform ...

11º - Move the cube to ARObject

12º - Assign ARObject to objectAR Layer, hit "Yes, Change Children"

13º - Move The ARobject away to the ARCamera for better visualization

14º - Rotate the ARobject to x=90 / y=180 / z = 0 , for better visualization on the editor ( thats how object will stay on tracked )
z vector downwards , x vector horizontal , and y vector will look at que camera

We have the object being tracked , but its not on the right position / scale


Lets Organize the things a bit ...


1º - Scale ARVirtualObject to x = 20/ y = 20 / z = 20

2º - Scale ARCamera and ARObject to x = 0.05 / y = 0.05 / z = 0.05

Now we start to see the object being tracked almost on the right position from the marker... Notice that if you move the marker around , it looks like the cube is moving on the marker ...

To fix that , we have to calibrate the cameras Field of view, you will have to play around with this values:

1º - ARCamera Field of View
2º - Plane (from background) position 

for this example, change ARCamera Field of view for 45 and it shold be fine ( if your plane is on the same place / scale from mine )
see now how the cubo remains over the sheep ...

Keep rotating the marker and moving from one side of screen to the other and see if it remains on the same place ( from marker ) thats where it should be... if its not , keep changing the values

Here I just create another cube and make a 'plane' for better visualization... I recommend you to do this, make a basic cube that indicate the size of the marker and another showing the rotation... To avoid losing  the marker position, you can always uncheck the mesh renderer if you don't want to view.

I just duplicated the cube , moved and scaled a bit... See now it has the almost same size from the marker, but its not on the center of the marker.

Go to ARObject and change the Shift: 

x = -0,07
y = 0,85
z = 0,14

See that now it has the same size and its centered = )

ps.:: It's not on the exactly right field of view ... you have to find the right value ( 48 / 48,5 it's a good start on this example )

The project will be here to download as soon as I discover how to attach it here, sorry.


And you can now start your Augmented Reality Apps...

obs : if you build the project it will not work ... but it will be fixed soon, do your things on the playmode for now


I would like to Thanks to RApp´s Studio ( for "free" me some time and space so I could start on Perceptual

Thanks to my Girlfriend for all the support s2 Love you xuxu s2

For more complete information about compiler optimizations, see our Optimization Notice.


Pruthviraj N.'s picture

Hi Pedro,

I am new to Unity and i tried your detailed example.

i am able to produce the cube.. but it doesnt stick with the sheep.. the cube remains at one place even if the cover page is moved around.. no translation

Kindly help,

Thanks a lot!

Pruthviraj N.'s picture

Thanks so much!!

I have been wandering all around and was worth it! :)

summer f.'s picture

Thank u so much !
that was great !!!!

I have a question !
( sorry if sounds stupid )

can we add UI in it ?
for selecting the shape and the color of the 3d model .
thanks .

Abhishek 81's picture

Great Info and awesome article!! Thanks!!

Add a Comment

Have a technical question? Visit our forums. Have site or software product issues? Contact support.