Default Behavior
When you protect existing behavior via automated testing, tentatively, you may sometimes want to just check whether the method is called or not. For example, I think there is the scene as follows: "I want to detect the invocation of any methods of the class Environment, because I want to know that whether the target method accesses the information of current environment, platform and so on." To achieve it, Prig supports the feature to modify the default behavior.
For example, imagine that there is a specification change against the following "good old-fashioned" code -- Of course, it has no test code 😨
public struct CommunicationContext
{
public int VerifyRuntimeVersion(string[] args)
{
...(snip)...
}
public int VerifyPrereqOf3rdParty(string[] args)
{
...(snip)...
}
public int VerifyUserAuthority(string[] args)
{
...(snip)...
}
public int VerifyProductLicense(string[] args)
{
...(snip)...
}
}
public static class JobManager
{
public static void NotifyStartJob(CommunicationContext ctx, string[] args)
{
int err = 0;
Log("Start prerequisites verification.");
if ((err = ctx.VerifyRuntimeVersion(args)) != 0)
goto fail;
if ((err = ctx.VerifyPrereqOf3rdParty(args)) != 0)
goto fail;
if ((err = ctx.VerifyUserAuthority(args)) != 0)
goto fail;
if ((err = ctx.VerifyProductLicense(args)) != 0)
goto fail;
Log("End prerequisites verification.");
Log("Start parameter construction.");
int mode = 0;
// ... Imagine that there are many instructions to calculate 'mode'...
if (err != 0)
goto fail;
bool notifiesError = false;
// ... Imagine that there are many instructions to calculate 'notifiesError'...
if (err != 0)
goto fail;
string hash = null;
// ... Imagine that there are many instructions to calculate 'hash'...
if (err != 0)
goto fail;
Log(string.Format("End parameter construction. " +
"code: {0}, mode: {1}, notifiesError: {2}, hash: {3}", err, mode, notifiesError, hash));
UpdateJobParameterFile(err, mode, notifiesError, hash);
return;
fail:
Log(string.Format("Notification failed. code: {0}, {1}:{2} at {3}",
err, Environment.MachineName, Environment.CurrentDirectory, Environment.StackTrace));
UpdateJobParameterFile(err, 0, false, null);
}
public static void UpdateJobParameterFile(int code, int mode, bool notifiesError, string hash)
{
...(snip)...
}
static void Log(string msg)
{
...(snip)...
}
}NotifyStartJob verifies the prerequisites to run a job at the beginning, then generates the parameter file to pass to the job if all prerequisites are satisfied. The way that uses 'goto' statement and goes to the common error procedure is standard way we used to see back in the times that C is mainstream. Even this day that notification by an exception is standard, the way might be used if the error is not real exception but just a domain specific error, or it will be performance issue.
You decided writing test code for it because you felt uneasy. At first, you protect its behavior to the extent possible by traditional mocking framework and a little bit of modification. When you look around the product code, CommunicationContext seems that it doesn't need to be struct. You change it to class, then you make its members are virtual:
public class CommunicationContext
{
public virtual int VerifyRuntimeVersion(string[] args)
{
...(snip)...
}
public virtual int VerifyPrereqOf3rdParty(string[] args)
{
...(snip)...
}
public virtual int VerifyUserAuthority(string[] args)
{
...(snip)...
}
public virtual int VerifyProductLicense(string[] args)
{
...(snip)...
}
}You want to abolish that the members are static, but it can't. Because the members are referenced from many other members. As a last resort, you make indirection stubs by using Prig here. In the Solution Explorer, right click JobManager's assembly reference and select Add Prig Assembly to add the Stub Settings File, then...:
you run the following command to copy the setting of UpdateJobParameterFile to clipboard. And you add it to the Stub Settings File(e.g. DefaultBehaviorSample.v4.0.30319.v1.0.0.0.prig):
PM> Add-Type -Path <full path to JobManager's assembly. e.g. "C:\Users\User\Prig.Samples\03.DefaultBehavior\DefaultBehavior\bin\Debug\DefaultBehavior.dll">
PM> Find-IndirectionTarget ([<full name to JobManager. e.g. DefaultBehavior.JobManager>]) UpdateJobParameterFile | Get-IndirectionStubSetting | Clip<?xml version="1.0" encoding="utf-8"?>
<configuration>
<configSections>
<section name="prig" type="Urasandesu.Prig.Framework.PilotStubberConfiguration.PrigSection, Urasandesu.Prig.Framework" />
</configSections>
<prig>
<stubs>
<!--
PJobManager.UpdateJobParameterFileInt32Int32BooleanString().Body =
(code, mode, notifiesError, hash) =>
{
throw new NotImplementedException();
};
-->
<add name="UpdateJobParameterFileInt32Int32BooleanString" alias="UpdateJobParameterFileInt32Int32BooleanString">
<RuntimeMethodInfo xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns:x="http://www.w3.org/2001/XMLSchema" z:Id="1" z:FactoryType="MemberInfoSerializationHolder" z:Type="System.Reflection.MemberInfoSerializationHolder" z:Assembly="0" xmlns:z="http://schemas.microsoft.com/2003/10/Serialization/" xmlns="http://schemas.datacontract.org/2004/07/System.Reflection">
<Name z:Id="2" z:Type="System.String" z:Assembly="0" xmlns="">UpdateJobParameterFile</Name>
<AssemblyName z:Id="3" z:Type="System.String" z:Assembly="0" xmlns="">DefaultBehavior, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null</AssemblyName>
<ClassName z:Id="4" z:Type="System.String" z:Assembly="0" xmlns="">DefaultBehavior.JobManager</ClassName>
<Signature z:Id="5" z:Type="System.String" z:Assembly="0" xmlns="">Void UpdateJobParameterFile(Int32, Int32, Boolean, System.String)</Signature>
<Signature2 z:Id="6" z:Type="System.String" z:Assembly="0" xmlns="">System.Void UpdateJobParameterFile(System.Int32, System.Int32, System.Boolean, System.String)</Signature2>
<MemberType z:Id="7" z:Type="System.Int32" z:Assembly="0" xmlns="">8</MemberType>
<GenericArguments i:nil="true" xmlns="" />
</RuntimeMethodInfo>
</add>
</stubs>
</prig>
</configuration>You can probably write a full test by the above preparation. For example, the following case can implement I think:
[TestFixture]
public class JobManagerTest
{
[Test]
public void NotifyStartJob_should_verify_using_the_methods_of_CommunicationContext_then_UpdateJobParameterFile()
{
using (new IndirectionsContext())
{
// Arrange
var mockCtx = new Mock<CommunicationContext>();
mockCtx.Setup(_ => _.VerifyRuntimeVersion(It.IsAny<string[]>())).Returns(0);
mockCtx.Setup(_ => _.VerifyPrereqOf3rdParty(It.IsAny<string[]>())).Returns(0);
mockCtx.Setup(_ => _.VerifyUserAuthority(It.IsAny<string[]>())).Returns(0);
mockCtx.Setup(_ => _.VerifyProductLicense(It.IsAny<string[]>())).Returns(0);
var ctx = mockCtx.Object;
var args = new[] { "foo", "bar", "baz", "qux" };
var mockUpdateJobParameterFile = new Mock<IndirectionAction<int, int, bool, string>>();
PJobManager.UpdateJobParameterFileInt32Int32BooleanString().Body = mockUpdateJobParameterFile.Object;
// Act
JobManager.NotifyStartJob(ctx, args);
// Assert
mockCtx.Verify(_ => _.VerifyRuntimeVersion(args), Times.Once());
mockCtx.Verify(_ => _.VerifyPrereqOf3rdParty(args), Times.Once());
mockCtx.Verify(_ => _.VerifyUserAuthority(args), Times.Once());
mockCtx.Verify(_ => _.VerifyProductLicense(args), Times.Once());
mockUpdateJobParameterFile.Verify(_ => _(0, 0, false, null), Times.Once());
}
}
}By the way, this time's specification change was "Remove the verification by license". In particular, you confirm to implement "VerifyProductLicense never be called". You modify the part of Assert of the test as the follows, and you confirm that test is failed:
...(snip)...
// Assert
mockCtx.Verify(_ => _.VerifyRuntimeVersion(args), Times.Once());
mockCtx.Verify(_ => _.VerifyPrereqOf3rdParty(args), Times.Once());
mockCtx.Verify(_ => _.VerifyUserAuthority(args), Times.Once());
// VerifyProductLicense will never be called, so change verification Once() -> Never().
mockCtx.Verify(_ => _.VerifyProductLicense(args), Times.Never());
mockUpdateJobParameterFile.Verify(_ => _(0, 0, false, null), Times.Once());
...(snip)...
All that's left is that you change the product code to passing the test code! You probably feel easy, so you modify JobManager as the follows, and you confirm that the test is passed. Easy task, isn't it?:
...(snip)...
public static void NotifyStartJob(CommunicationContext ctx, string[] args)
{
int err = 0;
Log("Start prerequisites verification.");
if ((err = ctx.VerifyRuntimeVersion(args)) != 0)
goto fail;
if ((err = ctx.VerifyPrereqOf3rdParty(args)) != 0)
goto fail;
if ((err = ctx.VerifyUserAuthority(args)) != 0)
goto fail;
// No longer need checking product license.
//if ((err = ctx.VerifyProductLicense(args)) != 0)
goto fail;
Log("End prerequisites verification.");
Log("Start parameter construction.");
int mode = 0;
...(snip)...
UpdateJobParameterFile(err, mode, notifiesError, hash);
return;
fail:
Log(string.Format("Notification failed. code: {0}, {1}:{2} at {3}",
err, Environment.MachineName, Environment.CurrentDirectory, Environment.StackTrace));
UpdateJobParameterFile(err, 0, false, null);
}
......Oh, my god 😱 This is an atrocity. Do you notice an issue? This always goes to goto fail;, and the parameter file that is passed to a job will be generated! In addition, unfortunately, JobManager sets err as code to UpdateJobParameterFile, so the job will be always run in the worst case. It recalls the worst-ever issue Behind iPhone's Critical Security Bug, a Single Bad 'Goto'.
To prevent these issues, I think that an exception should be thrown when any methods are called unintentionally. In this example, there are the procedure that writes the information of the environment that had a problem to log in the labeled fail. It seems that each property of Environment is retrieved. Therefore, throwing an exception is better when they are called. Environment is a class belonging in mscorlib, and the most members are static. In this case, you have no choice but to make its indirection stubs by using Prig. Right click Reference in the Solution Explorer and select Add Prig Assembly for mscorlib:
When you run the above command, you can make the Stub Settings File for mscorlib. Then, let's make the Prig Type for the public static members of Environment(paste the results of the following command to mscorlib.v4.0.30319.v4.0.0.0.prig):
PM> [System.Environment].GetMembers([System.Reflection.BindingFlags]'Public, Static') | ? { $_ -is [System.Reflection.MethodInfo] } | Get-IndirectionStubSetting | ClipNow, you can modify the default behavior against the public static members of Environment at one time. Add the following Arrange to the test code:
...(snip)...
[Test]
public void NotifyStartJob_should_verify_using_the_methods_of_CommunicationContext_then_UpdateJobParameterFile()
{
using (new IndirectionsContext())
{
// Arrange
// Set default behavior for most methods of Environment.
PEnvironment.
ExcludeGeneric().
// Environment.CurrentManagedThreadId is used by Mock<T>.Setup<TResult>(Expression<Func<T, TResult>>).
Exclude(PEnvironment.CurrentManagedThreadIdGet()).
// Environment.OSVersion is used by Times.Once().
Exclude(PEnvironment.OSVersionGet()).
DefaultBehavior = IndirectionBehaviors.NotImplemented;
var mockCtx = new Mock<CommunicationContext>();
mockCtx.Setup(_ => _.VerifyRuntimeVersion(It.IsAny<string[]>())).Returns(0);
mockCtx.Setup(_ => _.VerifyPrereqOf3rdParty(It.IsAny<string[]>())).Returns(0);
...(snip)...Also you have to take a little effort to exclude some members of Environment, because they are sometimes reserved depending on the mocking framework you use(In the Moq case, it is Environment.CurrentManagedThreadId and Environment.OSVersion). However, now you can watch that any methods are called unintentionally. In the above modification against NotifyStartJob, you can understand that the test is no longer passed. You will get real safe here!
For your information, when you write test cases, this feature enables that you confirm the progress of mocking, and you use as the guard that protects from external access under the complex condition. Also, you can change the default behavior to be returning default value if IndirectionBehaviors.DefaultValue is set. In a legacy code, I think that there are many scenes that the methods with the signature like void Foo() are called many times at one method. However, if you set their behavior to be do-nothing except for the part of testing, you can probably narrow down the point of view more easily.
Complete source code is here