среда, 27 июля 2011 г.

The BDD scenario problems and solutions (for developers) meeting result

Recently we've acquired 4 opened issues connected with BDD-style development

Here they are:

The solutions were developed.

Too detailed requirements

We can solve it with macro commands.

Macro commands are just several action steps combined in one name.

In example, instead of writing

Given process 'All Practices' defined
And project 'TargetProcess' for process 'All Practices' created

We will be able to write:

Given TargetProcess project created

So, we need to be able to define an action step (macro command) which consists actually defined with using of two other action steps.

No test output will be hurt! The output will be just macro command, not it's internal action steps.

Too Many Action Steps

Actually, having many action steps is not a problem. The real problem is that we can't navigate through them during scenario writing.

It was decided to try to implement intellisence for bdd specs.

Badly Structured Specs Location

Indeed, it's quite hard to switch from unit tests organization (class-level) to bdd specs (feature-level).


We need to switch our structure to provided above.

среда, 20 июля 2011 г.

BDD Specs Meeting Followup

Today we discussed several issues connected with BDD specs and their role in QA and Dev Team collaboration.

As the resule the following agreements were achieved:

  1. No User Story should be implemented or taken in In Progress state without specs (QA + Dev Team + Product Owner)
  2. User Story specs should be updated with possible BDD scenarious. Those specs should be introduced as Test Cases (QA)
  3. Specs should be revised by Dev Team and updated if necessary (QA+Dev Team)
  4. Specs (formed Test Cases) should be marked as Passsed when implemented (Dev Team)

понедельник, 18 июля 2011 г.

Feature toggling in TP

Recently some feature toggling utilities were introduced in TargetProcess.

All the features are enumerated at TpFeature enum:

public enum TpFeature
{
None = 0,
NewPopEmailIntegration,
TestCaseLibrary,
Solution,
...
}

Features are toggled on/off in web.config
<TpFeatures>
<add Feature="NewPopEmailIntegration" />
<!--<add Feature="Solution" />-->
<!--<add Feature="TestCaseLibrary" />-->
<!--<add Feature="UserStoriesCountReport" />-->
<!--<add Feature="BugCountReport" />-->
</TpFeatures>


Here are some use cases with samples.
1. I want to ignore some unit tests failed because I turned off the feature they're testing.
[Test]
[ToggleFeatureOff(TpFeature.Solution)]
public virtual void AttachToSolution()
{
...
}

It seems that attribute toggles tests on or off depending on feature configuration but unfortunately it’s not so clever. The attribute inherits nUnit IgnoreAttribute and is just descriptive to make the reason of ignorance more clear. This also will cause compile time errors when I'll be removing TpFeature member to fully remove or integrate feature. So, I won't forget about them.

2. I want to exclude some part of markup from page if feature is toggled off and include it when toggled on.
<tp:Toggle runat="server" Feature="Solution">
<tp:EditLink ID="lnkAttach" runat="server" TypeOfEntity="Request" Text="Attach To Solution" ClientAction='<%#string.Format("ShowSolutionFinder({0},{1})", Eval("RequestID"), Eval("ProjectID"))%>' />
<asp:Literal ID="separator3" runat="server">|</asp:Literal>
</tp:Toggle>


3.I want to include some part of markup to page when feature is toggled on and include another part of markup when feature is toggled off.
<tp:Toggle ID="newEmailIntegration" Feature="NewPopEmailIntegration" TurnedOn="true" runat="server">
<p class="defaultLabel pb-5">
Inbound Email Integration
</p>
<tp:InboundMailReplyEmail ID="tpInboundMailReplyEmailNew" runat="server" />
</tp:Toggle>
...
<tp:Toggle ID="oldEmailIntegration" Feature="NewPopEmailIntegration" TurnedOn="false" runat="server">
...
</tp:Toggle>


4. I want to hide some help steps (from Library.xml) when feature is toggled off.
<Step Key="BugsByState" Dependency="Project" Practice="BugTracking" Action="List" EntityType="Bug" Feature="BugCountReport">
<ActionItem ActionID="seeBugsDistributionByState" />
</Step>
<Step Key="BugsBySeverity" Dependency="Project" Practice="BugTracking" Action="List" EntityType="Bug" />
<Step Key="BugsProgress" Dependency="Project" Practice="BugTracking" Action="List" EntityType="Bug" Feature="BugCountReport"/>


5. I want to exclude some sitemap nodes when feature is toggled off
<siteMapNode title="Test Case Library" url="~/..." action="List" entity="TestCase" key="TestCaseLibrary" feature="TestCaseLibrary">
<siteMapNode title="Add/Edit Test Case" url="~/..." action="Edit" entity="TestCase" key="TestCaseLibrary" />
</siteMapNode>


6. I want to check if feature is enabled at any place in source code.
if (ObjectFactory.GetInstance<ITpFeatureList>().IsEnabled(TpFeature.Solution))
{
yield return GetDescription(EmailTemplateName.RequestToSolutionAttached);
}

Visual Builds Board



<jenkinsDashboardSection uri="http://jenkinsmaster-hv:8080" interval="0:05:00">
              <forceBuild
                     uri="http://jenkinsmaster-hv:8080/job/BuildBranch/build?delay=0sec"
                     body="name=BRANCH_TO_BUILD&amp;value=$1&amp;json=%7B%22parameter%22%3A+%7B%22name%22%3A+%22BRANCH_TO_BUILD%22%2C+%22value%22%3A+%22$1%22%7D%7D&amp;Submit=Build" />
              <avatars users="http://plan.tpondemand.com/api/v1/Users?include=[id,email]&amp;take=1000&amp;token=zzzzzzzzzzzzzz" avatar="http://plan.tpondemand.com/avatar.ashx?size=32&amp;UserId={0}"/>
              <branchTitle
                     namePattern="(?:US|Bug)#?(?&lt;number&gt;\d+).*"
                     titlePattern="\[&quot;(.*?)&quot;\]"
                     title="$1"
sourceUri="http://plan.tpondemand.com/api/v1/Assignables/${number}?include=[Name]&amp;format=array&amp;token=zzzzzzzzzzzzzzz"               externalUri="http://plan.tpondemand.com/View.aspx?ID=${number}"
                     />
              <!---->
              <monitoredJobs>
                     <job name="BuildAll"/>
                     <job name="BuildCommit"/>
                     <job name="BuildBranchUnitTestsOnly"/>
                     <job name="BuildBranch"/>
                     <job name="FastBuildBranch"/>
                     <job name="NightlyMasterBuild"/>
              </monitoredJobs>
              <buildParts>
                     <part name="Unit Tests">
                           <job name="BuildUnitTests" />
                           <job name="UnitTestPart0"/>
                           <job name="UnitTestPart1"/>
                           <job name="UnitTestPart2"/>
                           <job name="UnitTestPart3"/>
                           <job name="UnitTestPart4"/>
                           <job name="UnitTestPart5"/>
                           <job name="UnitTestPart6"/>
                           <job name="UnitTestPart7"/>
                           <job name="UnitTestPartOther"/>
                     </part>
                     <part name="Plugins" mode="FailOnly">
                           <job name="BuildPluginsUnitTests" />
                     </part>
                     <part name="Build" mode="FailOnly"  aggregation="Optimistic">
                           <job name="BuildPackage" />
                           <job name="BuildApplication" />
                     </part>
                     <part name="F00" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestPart0"/>
                           <job name="FastFuncTestPart0"/>
                     </part>
                     <part name="F01" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestPart1"/>
                           <job name="FastFuncTestPart1"/>
                     </part>
                     <part name="F02" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestPart2"/>
                           <job name="FastFuncTestPart2"/>
                     </part>
                     <part name="F03" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestPart3"/>
                           <job name="FastFuncTestPart3"/>
                     </part>
                     <part name="F04" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestPart4"/>
                           <job name="FastFuncTestPart4"/>
                     </part>
                     <part name="F05" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestPart5"/>
                           <job name="FastFuncTestPart5"/>
                     </part>
                     <part name="F06" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestPart6"/>
                           <job name="FastFuncTestPart6"/>
                     </part>
                     <part name="F07" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestPart7"/>
                           <job name="FastFuncTestPart7"/>
                     </part>
                     <part name="F08" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestPart8"/>
                           <job name="FastFuncTestPart8"/>
                     </part>
                     <part name="F09" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestPart9"/>
                           <job name="FastFuncTestPart9"/>
                     </part>
                     <part name="F10" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestPart10"/>
                           <job name="FastFuncTestPart10"/>
                     </part>
                     <part name="F11" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestPart11"/>
                           <job name="FastFuncTestPart11"/>
                     </part>
                     <part name="F12" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestPart12"/>
                           <job name="FastFuncTestPart12"/>
                     </part>
                     <part name="F13" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestPart13"/>
                           <job name="FastFuncTestPart13"/>
                     </part>
                     <part name="FOt" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestPartOther"/>
                           <job name="FastFuncTestPartOther"/>
                     </part>
                     <part name="FI" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestIntegration"/>
                           <job name="FastFuncTestPartIntegration"/>
                     </part>
                     <part name="FV" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestPartViews"/>
                           <job name="FastFuncTestPartViews"/>
                     </part>
                     <part name="FP1" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestPluginsPart1" />
                           <job name="FastFuncTestPluginsPart1" />
                     </part>
                     <part name="FP2" mode="FailOnly" aggregation="Optimistic">
                           <job name="FuncTestPluginsPart2" />
                           <job name="FastFuncTestPluginsPart2" />
                     </part>


                     <part name="UFT" mode="Unimportant">
                           <job name="FuncTestPartUnstable"/>
                           <job name="FastFuncTestPartUnstable"/>
                           <job name="FuncTestPluginsUnstable"/>
                           <job name="FastFuncTestPluginsUnstable"/>
                     </part>
                     <part name="CC" mode="Unimportant">
                           <job name="CodeCoverage"/>
                     </part>
              </buildParts>
       </jenkinsDashboardSection>



Xpath Description Sample
@uri Root uri for Jenkins http://jenkinsmaster-hv:8080
@interval Pool interval 0:05:00
forceBuild/@uri Link for "Force Build" button (actually, run the specific job with given parameters) http://jenkinsmaster-hv:8080/job/BuildBranch/build?delay=0sec
forceBuild/@body Body of POST request for "Force Build button". $1 is a placeholder for the build name
avatars  Section to get avatars for changes (now are hardcoded for @TargetProcess.com domain)
branchTitle Settings to get branch descriptions from TP
branchTitle/@namePattern Regex for Git branch name (?:US|Bug)#?(?&lt;number&gt;\d+).*
branchTitle/@sourceUri Uri to TP Rest api to get an assignable name. Use placeholders defined in @namePattern http://plan.tpondemand.com/api/v1/Assignables/${number}?include=[Name]&amp;format=array&amp;token=zzzz"
branchTitle/@titlePattern How to get title from @sourceUri response \[&quot;(.*?)&quot;\]
branchTitle/@title  Title to display. Use placeholders from @titlePattern $1
branchTitle/@externalUri Uri for links on title. Use placeholders from @namePattern http://plan.tpondemand.com/View.aspx?ID=${number}
monitoredJobs/job/@name List of root jobs to monitor
buildParts/part Definition of single bubble on a build
buildParts/part/@name Name of bubble
buildParts/part/job/@name Jobs to include to the bubble
buildParts/part/@mode  How bubble change the color of entire build (FailOnly - only red color is used, Unimportant - do not use the bubble color, Normal (default) - use red and yellow color).
buildParts/part/@aggregation How bubble color is calculated. Optimistic - use "the best" status. Pessimistic (default) - use "the worst" status. 

The dashboard is designed to show hierarchic jobs. For example, in our case we have the following hierarchy:


  • BuildAll
    • BuildUnitTests
      • UnitTestsPart1
      • ...
      • UnitTestsPart7
    • BuildApplication
      • FastFuncTestPart1
      • ...
      • FastFuncTestPart15
    • BuildPacakage