CruiseControl.Net

[Andrew]

I have a UI Test project that belongs to the same solution as the project it tests.

when I check my code into subversion our build server will kick off, and I get this error:

D:\CI\eZone_Working\eZoneUITests\eZoneUITests.csproj(91,3): error MSB4019: The imported project "C:\Program Files (x86)\MSBuild\Telerik\WebUITestStudio\Telerik.WebUITestStudio.targets" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.

D:\CI\eZone_Working\eZoneUITests\eZoneUITests.csproj(91,3): error MSB4019: The imported project "C:\Program Files (x86)\MSBuild\Telerik\WebUITestStudio\Telerik.WebUITestStudio.targets" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.

Failed to start MSBuild.

External Program Failed: C:\Windows\Microsoft.NET\Framework\v4.0.30319\msbuild.exe (return code was 1)

I am not trying to run the ui test at this point, but it has stopped the CruiseControl from being able to build the project.
what do I need to do to get this server setup correctly to handle solutions that have a UI test projects in them so they can build?

[Cody]

Hello Andrew,

This is basically being caused by this line contained in the .csproj file:

<Import Condition="true" Project="$(MSBuildExtensionsPath)\Telerik\WebUITestStudio\Telerik.WebUITestStudio.targets" />

This line is telling MSBuild to load our build extension which is what generates the Pages.g.cs/vb file that goes into compiling a test project that contains Test Studio tests. The error message is telling you that MSBuild cannot find our build extension... probably because Test Studio is not installed on that build server.

To overcome this problem you need to purchase and install a copy of our Run-Time edition which sells for just $199.
All the best,
Cody
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

[Andrew]

thanks, we have Ultimate Collection I will install it and see if that fixes it

[Cody]

Hello Andrew,

The Ultimate Collection is basically a set of installers and our licensing allows you to install them on only one machine at a time. Thus you will consume one full license to all components of your Ultimate Collection by installing it on your build server. That sounds like a heavy price to pay to get Test Studio builds to work on your build server (unless it's a license you don't need elsewhere)

Best wishes,
Cody
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

[Andrew]

ya you are right, I thought about that after... I need to talk to the higher up's to see about getting a license, but you guys should give away a runtime version that will not let you build test just run them.

[Andrew]

according to this page
http://www.telerik.com/products/test-studio-express.aspx

where it talks about "Continuous integration" it does not say anything about having to purchase another product to do this, we upgrade all of our licenses to the Utilmate Collection mainly for the Testing Software and that it would work with out CI server out of the box, now we are being told we have to buy another product? what's with that? we feel like be are being nickeled and dimed here.

[Daniel Levy]

Hi Andrew,

I see understand your frustration and can assure you that we value you and all our customers and will correct the wording on our website.

 I'd like to take this conversation offline - I will send you an email shortly.

Greetings,
Daniel Levy
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

[Andrew]

again thanks for your help.

I have installed the software, but I msut be missing something here I still get this error from cruise control

C:\Program Files (x86)\MSBuild\Telerik\WebUITestStudio\Telerik.WebUITestStudio.targets(9,5): error MSB4062: The "ArtOfTest.WebAiiVSIP.CodeGeneration.GenerateElementsTask" task could not be loaded from the assembly C:\Program Files (x86)\MSBuild\Telerik\WebUITestStudio\PrivateAssemblies\ArtOfTest.WebAiiVSIP.dll. Could not load file or assembly 'file:///C:\Program Files (x86)\MSBuild\Telerik\WebUITestStudio\PrivateAssemblies\ArtOfTest.WebAiiVSIP.dll' or one of its dependencies. The system cannot find the file specified. Confirm that the <UsingTask> declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask. [D:\CI\eZone_Working\eZoneUITests\eZoneUITests.csproj]

C:\Program Files (x86)\MSBuild\Telerik\WebUITestStudio\Telerik.WebUITestStudio.targets(9,5): error MSB4062: The "ArtOfTest.WebAiiVSIP.CodeGeneration.GenerateElementsTask" task could not be loaded from the assembly C:\Program Files (x86)\MSBuild\Telerik\WebUITestStudio\PrivateAssemblies\ArtOfTest.WebAiiVSIP.dll. Could not load file or assembly 'file:///C:\Program Files (x86)\MSBuild\Telerik\WebUITestStudio\PrivateAssemblies\ArtOfTest.WebAiiVSIP.dll' or one of its dependencies. The system cannot find the file specified. Confirm that the <UsingTask> declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask. [D:\CI\eZone_Working\eZoneUITests\eZoneUITests.csproj]

Failed to start MSBuild.

External Program Failed: C:\Windows\Microsoft.NET\Framework\v4.0.30319\msbuild.exe (return code was 1)

again I am not trying to run a test at this point just trying to get the build server to build a solution that has a telerik ui test project included in it

[Cody]

Hello Andrew,

I'd like to look at this problem on your computer via GoToMeeting. Let me know what time zone you are in and your availability and I'll setup the meeting. I'm in Austin, TX which is Central Time (GMT -6).

Kind regards,
Cody
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

[Andrew]

I am eastern time,I am available tomorroww between 11 -  1 or anytime on thursday between  9  -  4

[Cody]

Hello Andrew,

Just to summarize the outcome of our GoToMeeting today... we discovered that you do not have a full version of Visual Studio installed on your CC.NET build server. As a result the Test Studio installer did not install the support for compiling Test Studio test projects via MSBuild.

Right now today you have two choices:
1) Install a full version of Visual Studio (Professional edition will suffice) then re-install Test Studio Run-Time
2) Split out the Test Studio project into it's own VS solution. Don't ask CC.NET to build it. Instead execute the test via our Scheduling server or using our ArtOfTest.Runner as a task in your CC.NET build script. 

I have taken this feedback and forwarded it to our product manager as a feature request. Hopefully someday in the not too distant future we can add support for your current environment.

Best wishes,
Cody
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

[Andrew]

Hi Cody,

Ok, I installed VS 2010 Pro and the project is now building. I do get this warning but I am sure it is not an issue:
C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Common.targets(1360,9): warning MSB3247: Found conflicts between different versions of the same dependent assembly. [D:\CI\eZone_Working\eZoneUITests\eZoneUITests.csproj]

anyways I have 2 other guestions.

1. Whenever someone logs into our build server we get a UAC message asking "Do you want to allow the following program from an unknown publisher to make changes to this computer"

Program name: Telerik.TestStudio.Scheduling.Setup.exe

2. do you have an exaple of how I can get ccnet to run a test after a build?

[Cody]

Hello Andrew,

Chances are the MSB3247 error is the result of having both Visual Studio and "Microsoft Visual C++ Compiler" installed at the same time.

For item 1) if you have no plans to use our Scheduling Server you can uncheck that feature during the install of the Run-Time. Then that feature won't get installed and it will stop that warning message.

For item 2) this KB article should help.

I have filed a feature request here about not requiring full Visual Studio to be able to build Test Studio projects in your environment.

Greetings,
Cody
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

[Andrew]

thanks, I think I am close but I am still having an issue.
here is what I gotin my ccnet config :

<exec>
           <!--Delete the test Results file first. This is required as MsTest will not create the file if it exists this could be merged with the mstest action in a single batch file-->

                    <executable>$(windir)\system32\cmd</executable>
                    <baseDirectory>D:\CI\eZone_Working\eZoneUITests</baseDirectory>
                    <buildArgs>/c if exist Faculty_TestResults.trx del Faculty_TestResults.trx /f</buildArgs>
                    <buildTimeoutSeconds>30</buildTimeoutSeconds>
                </exec>
            <exec>
                    <!-- Call mstest to run the tests contained in the TestProject -->
                    <executable>C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\mstest.exe</executable>
                    <baseDirectory>D:\CI\eZone_Working\eZoneUITests</baseDirectory>

                    <!-- testcontainer: points to the DLL that contains the tests -->
                    <!-- runconfig: points to solutions testrunconfig that is created by vs.net, list what test to run -->
                    <!-- resultsfile: normally the test run log is written to the uniquely named testresults directory  -->
                    <!-- this option causes a fixed name copy of the file to be written as well-->
                    <buildArgs>/testcontainer:bin\debug\eZoneUITests.dll /test:Faculty /resultsfile:Faculty_TestResults.trx</buildArgs>
                    <buildTimeoutSeconds>30</buildTimeoutSeconds>
                </exec>
        </tasks>
        <publishers>
                <!--to get the test results in the dashboard we have to merge the results XML file -->
                <merge>
                    <files>
                        <file>D:\CI\eZone_Working\eZoneUITests\Faculty_TestResults.trx</file>
                    </files>
                </merge>
                <xmllogger />
            </publishers>

and here is what I get from the build log:

<buildresults>
  <message>Microsoft (R) Test Execution Command Line Tool Version 10.0.30319.1</message>
  <message>Copyright (c) Microsoft Corporation. All rights reserved.</message>
  <message>    </message>
  <message>Loading bin\debug\eZoneUITests.dll...</message>
  <message>Test Faculty cannot be found.</message>
  <message>Test Faculty cannot be found.</message>
  <message>Starting execution...</message>
  <message>No tests to execute.</message>
</buildresults>

my eZoneUITests project has 3 test files (Faculty.tstest, Staff.tstest, Student.tstest) i and just trying to run Faculty.tstest at this point but plan on running all of them and more.

[Cody]

Hi Andrew,

When using MSTest to execute tests you actually need to use a command line syntax like this:

mstest /testcontainer:.\GoogleSearch.tstest

Note you do NOT point to the .dll that the test will be using. If you create a Visual Studio test list you want to use this syntax:

mstest /testlist:SampleTestList /testmetadata:..\MSTest-Tutorial.vsmdi

This is documented here.

NOTE: MSTest cannot be used to execute a Test Studio test list (i.e. a xxxx.aiilist file).

All the best,
Cody
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

[Andrew]

is mstest the recomended way to run my ui tests from ccnet or is there another way?
I just want to make sure I am doing this the right way

[Andrew]

I tried the example you sent and it looks like the test is trying to run but I get an error with it

<buildresults>
  <message>Microsoft (R) Test Execution Command Line Tool Version 10.0.30319.1</message>
  <message>Copyright (c) Microsoft Corporation. All rights reserved.</message>
  <message>    </message>
  <message>Loading .\Faculty.tstest...</message>
  <message>.\Faculty.tstest</message>
  <message>Property accessor 'Name' on object 'ArtOfTest.WebAiiVSIP.WebAiiTest' threw the following exception:'Object reference not set to an instance of an object.'</message>
</buildresults>

This test does run in visual studio

[Cody]

Hi Andrew,

We neither recommend nor discourage anyone from using MSTest (with CC.NET or any other build system). It is simply one way of running tests that works just fine. There are other ways of running tests. Which you select depends on what is/isn't important to you. For example some customers like the ability to publish the results from a MSTest run back into TFS. For others they don't care (or don't have TFS).

Can you run the MSTest from a Visual Studio command prompt? Try manually running something like this at the command line and tell me what happens:

mstest /testcontainer:.\GoogleSearch.tstest

If that gives you problems I'd like to look at this problem directly on your computer via GoToMeeting.

Kind regards,
Cody
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

[Andrew]

I got the same error when run from the command prompt

[Andrew]

ok, I ran the test from post command prompts and it looks like it worked.

but I am now back to my build log saying this:

<buildresults>
  <message>Microsoft (R) Test Execution Command Line Tool Version 10.0.30319.1</message>
  <message>Copyright (c) Microsoft Corporation. All rights reserved.</message>
  <message>    </message>
  <message>Loading bin\debug\eZoneUITests.dll...</message>
  <message>Test Faculty cannot be found.</message>
  <message>Test Faculty cannot be found.</message>
  <message>Starting execution...</message>
  <message>No tests to execute.</message>
</buildresults>

[Cody]

Hello Andrew,

Would you share with me the <exec> section from your ccnet config so I can confirm it looks setup correctly?

Kind regards,
Cody
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

[Andrew]


<exec>

         <!-- Call mstest to run the tests contained in the TestProject -->

         <executable>C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\mstest.exe</executable>

         <baseDirectory>D:\CI\eZone_Working\eZoneUITests</baseDirectory>

  

         <!-- testcontainer: points to the DLL that contains the tests -->

         <!-- runconfig: points to solutions testrunconfig that is created by vs.net, list what test to run -->

         <!-- resultsfile: normally the test run log is written to the uniquely named testresults directory  -->

         <!-- this option causes a fixed name copy of the file to be written as well -->

 

         <buildArgs>/testcontainer:.\Faculty.tstest /resultsfile:eZone_TestResults.trx</buildArgs> -->

         <buildTimeoutSeconds>30</buildTimeoutSeconds>

</exec>

[Cody]

Hello Andrew,

I am confused by the results compared to the command line. What confuses me is this line from the results:

<message>Loading bin\debug\eZoneUITests.dll...</message>

This implies you used a command line like:

"MSTest /testcontainer:bin\debug\eZoneUITests.dll"

We do not want to point MSTest to the dll. We want to point it to the .tstest file, the way you have stated in your last message "/testcontainer:.\Faculty.tstest".

Would you mind trying it one more time? It just doesn't make sense you would get the results you got given that command line.

All the best,
Cody
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

[Andrew]

<exec>
                    <!-- Call mstest to run the tests contained in the TestProject -->
                    <executable>C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\mstest.exe</executable>
                    <baseDirectory>D:\CI\eZone_Working\eZoneUITests</baseDirectory>

                    <!-- testcontainer: points to the DLL that contains the tests -->
                    <!-- runconfig: points to solutions testrunconfig that is created by vs.net, list what test to run -->
                    <!-- resultsfile: normally the test run log is written to the uniquely named testresults directory  -->
                    <!-- this option causes a fixed name copy of the file to be written as well -->
                    <buildArgs>/testcontainer:.\Faculty.tstest /resultsfile:eZoneTestResults.trx</buildArgs> -->
                <buildTimeoutSeconds>30</buildTimeoutSeconds>
                </exec>

I ran the test again and still the same issue, the have noticed these is no results file being created but I guess that is because the test is not running

[Andrew]

ok,  got it to work.. but the tests is still failing here is the test results file

[Cody]

Hi Andrew,

I'm going to setup a CC.NET server here and see if I can figure out what's going on. Please give me until Monday to work on this.

Regards,
Cody
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

[Andrew]

any luck?

as we are running  ccnet as a service could this effect it?

[Cody]

Hi Andrew,

I apologize for the delay getting back to you. I finally got to the bottom of this. Turns out there's a very subtle bug in Test Studio that is causing the error:

Failed to queue test run <test run name here': Value cannot be null.
Parameter name: path1

I have filed a high priority bug here.

Fortunately there's a couple of easy work arounds.

Work around A:
1) Load the test in Test Studio
2) Add a coded step
3) Delete the coded step

This modifies the test definition such that it is now compatible with MSTest.

Work around B:
1) Load the test in Visual Studio
2) Make any change to the test
3) Undo the change (unless you meant to keep the change)
3) Save the test

This also modifies the test definition such that it is now compatible with MSTest.

Yes, running CC.NET as a service will cause a problem with dialog handling and/or any test steps that need to move the mouse or simulate typing at the keyboard. If your tests do any of these things you will need to run CC.NET via the command line.

Lastly I found that the configuration of a project in CruiseControl has change significantly since I wrote our documentation. I'll work on creating a new document showing how it's done in the current version. In short this is the project configuration I used on my config.xml:

<project name="Test_Studio_Project_A">

  <modificationset quietperiod="30">
    <!-- touch any file in Test_Studio_Project_A project to trigger a build -->
    <filesystem folder="projects/${project.name}"/>
  </modificationset>

  <schedule interval="10">
    <exec command="C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe"
         workingdir="projects/${project.name}"
         args="&quot;/testcontainer:C:\Program Files (x86)\cruisecontrol-bin-2.8.4\projects/${project.name}\WebTest Yahoo.tstest&quot;"
         />
  </schedule>
</project>

What I did was I placed my Test Studio project in the CruiseControl projects folder "C:\Program Files (x86)\cruisecontrol-bin-2.8.4\projects/" and named that folder "Test_Studio_Project_A". This is not required but it did allow me to use the built-in CruiseControl ${project.name} macro.

Regards,
Cody
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

[Andrew]

Thanks,

also we have ccnet running as a service under the local system with iteract with desktop on and I get this issue

------------------------------------------------------------

'24/10/2011 9:58:53 AM' - Starting execution....

------------------------------------------------------------

------------------------------------------------------------

'24/10/2011 9:58:55 AM' - LOG: Unexpected dialog encountered. Closing the dialog, and halting execution.

'24/10/2011 9:58:56 AM' - 'Pass' : 1. Navigate to : 'http://biz-devapps.ivey.ca/eZone/SiteLogin.aspx?ReturnUrl=%2fezone%2fdefault.aspx'

'24/10/2011 9:58:56 AM' - 'Fail' : 2. Wait for element 'ContentPlaceHolder1LgnLoginUserNameText' 'is' visible.

------------------------------------------------------------

Failure Information:

~~~~~~~~~~~~~~~

Unexpected dialog: Privacy

I am thinking this is cause IE has never been run under this account and is prompting for first run settings. do you know how to fix this.

sorry I am not sure what you mean here:

"Yes, running CC.NET as a service will cause a problem with dialog handling and/or any test steps that need to move the mouse or simulate typing at the keyboard"

is this when you have "SimulateRealClick" and "SimulateRealTyping" checked?

[Cody]

Hi Andrew,

First, PITS bug 8268 was just fixed. The fix will be included in our next internal build due out on Friday this week.

To answer your next question, yes "SimulateRealClick" and "SimulateRealTyping" is exactly what I'm referring to. Plus there's also the mouse actions as described in the middle of this documentation page.

No I am sorry I do not know how to fix the error you have posted. I am not aware of anyone successfully running Test Studio tests using CC.NET running as a service. That's the reason I instruct all my customers to run CC.NET via command line. I tell our TFS customers the same thing when they're trying to run Test Studio tests as part of their TFS builds.

Kind regards,
Cody
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

[Andrew]

Thanks,

I was just asking cause my boss, will just not accept the whole running ccnet from the command window and locking the machine and I have pointed out that any UI testing software would have the same issue and they all need to launch a browser to do there tests.

just one last question have you every tried to run the tests under Jenkins we are going to be moving to this CI software in the near future, I bet most of the settings will be the same as CCnet but I was just wondering if you have any info on it

Again thanks for all you help

[Cody]

Hello Andrew,

You are 100% about any form of UI tests running into this problem. Many of the managers express the same security concern. Our best recommendation is to use some sort of a VNC product and simply disconnect the VNC connection. This leaves the Windows session open and unlocked but you have to log back into the VNC connection before you can get back on the machine.

I believe I already answered your question about Jenkins in your other support ticket. I did forget to mention i can award you quite a few Telerik Points if you will write up a white paper similar to ours showing how to do it.

Regards,
Cody
the Telerik team

Check out Telerik Trainer, the state of the art learning tool for Telerik products.

[Darren]

I very much agree with this. This seems like a case of bait-and-switch. Now I'm going to have to go back to the CTO and tell him that we need to lay out more money to run it on CI.

[Cody]

Hi Darren,

I am sorry but I don't see how this is anything like "bait-and-switch". We have never stated we can run on a locked machine. In fact the post you are responding to is 3 years old and has been publicly visible that whole time. We've also had this online KB article for years that discusses the problem and possible work arounds.

For myself, I handle this situation by using Microsoft Hype-V (which requires authentication to access) and it's Manager console. I can connect and disconnect to any VM I need at will and my tests will still run while disconnected. It's a feature of the Hyper-V product. I suspect that VMWare has something similar. I just don't have it to use and test and confirm.

Regards,
Cody
Telerik