getting the valid input in mock

[dvu4]

Hi,

I tried to mock the input type of a function create_service in services.py .

from mockito import verify, when, ANY, not_
from .api import services
from .models import Token


class TestCreateServicePrincipal(TestCase):
	def test_input_name_service_principal(self):
		when(services).create_service(ANY(Token), ANY(str), None, None, "p",False).thenReturn(1)
		when(services).create_service(not_(ANY(Token)), ANY(str), None, None, "p",False).thenRaise(TypeError)
		when(services).create_service(ANY(Token), not_( ANY(str)), None, None, "p",False).thenRaise(TypeError)
		when(services).create_service(not_(ANY(Token)), not_( ANY(str)), None, None, "p",False).thenRaise(TypeError)
		verify(services).create_service(ANY(Token), ANY(str), None, None, "p",False)
                #verify(services).create_service(not_(ANY(Token)), ANY(str), None, None, "p",False)
                #verify(services).create_service(ANY(Token), not_( ANY(str)), None, None, "p",False).thenRaise(TypeError)
		#verify(services).create_service(not_(ANY(Token)), not_( ANY(str)), None, None, "p",False).thenRaise(TypeError)

I am setting up a mock with when and then using verify to evaluate the valid or invalid inputs but I got the following error. Is there any way I can fix the error?

mockito.verification.VerificationError: Wanted but not invoked: create_service(<Any: <class 'Token'>>, <Any: <class 'str'>>, None, None, 'p', False) Instead got: Nothing

[kaste]

I think there is a misconception here. You first when something and then immediately verify the usage of the mock but you actually never used the mock. T.i. there is no execution phase in the test. (when = setup, verify = assert, the execution of the system under test is missing)

[dvu4]

Can you explain more what is missing in the execution of the system under test?

[kaste]

Typically

when(services).create_service(...).thenReturn(1)  # setup
assert services.create_service(1, 2, 3, 4, 5, 6) == 1  # usage
# now you could verify; you don't need though because you already asserted
unstub()  #  clean up (undo the stubbing) after the test, usually in the tearDown()

[dvu4]

Thank you for clarification. Is that any way I can assert it raises the correct exception like TypeError. I tried with invalid Token class

data = {
		"token_type": "mock_token", 
		"expires_in": 2023, 
		"ext_expires_in": 2025, 
		"access_token": 1 # --> valid one should be str , I changed it to int
	    }
token = Token(**data)
long_name = "foo"
when(services).create_service(not_(ANY(Token)), ANY(str), None, None, "p",False).thenRaise(TypeError)  # setup
assert services.create_service(token, long_name, None, None, "p",False)  # usage

But I got the error

mockito.invocation.InvocationError: 
Called but not expected:

    create_service(Token(token_type='mock_token', expires_in=2023, ext_expires_in=2025, access_token='1'), 'foo', None, None, 'p', False)

Stubbed invocations are:

    create_service(<Not: <Any: <class 'models.Token'>>>, <Any: <class 'str'>>, None, None, 'p', False)

[kaste]

I still think there is something odd here. Are you experienced with mock driven development or TDD etc. ?
The when(services).create_serceive(not_(any_()),...) seems untypical specific. The matchers not_(ANY(...))) translate to if not isinstance(token, Token: ... raise TypeError but then you actually call it with a token so this never matches the expected/stubbed invocation. If you define one very specific when invocation, mockito expects and only responds iff the system under tests calls and uses that exact same call/invocation signature. Otherwise it throws "unexpected invocation".

E.g.

when(os.path).exists("foo").thenReturn("No")
# usage
os.path.exists("bar")   # <= this will throw

For your code, typically the real implementation of create_service will throw TypeError if it is called with the wrong arguments. I assume you don't want to execute the code in create_service. How does the code look you actually test here? You don't test create_service here but the function which calls create_service. Am I right?

I see that the token you instantiate is an invalid token, but it is an instance of Token. Who decides this token is invalid?

[dvu4]

I am not experienced with mock driven development or TDD. My main goal is to test if the input is valid or not. The instance Token is defined below,

class Token(BaseModel):
    token_type: str
    expires_in: int
    ext_expires_in: int
    access_token: Optional[str] = None

I don't want to execute the code in create_service. I just want to throw exception such as TypeError if input is incorrect.
I am not sure if mockito allows me to define the correct type such as when(services).create_serceive(not_(any_()),...) , then test if I add a specific input to test the validity of the input.

I am wrong when calling token with a Token in The matchers not_(ANY(Token))), so I changed token to a random string

	def test_invalid_token_valid_long_name(self):
		token = "mock_token"
		long_name = "foo"

		when(service).create_service(not_(ANY(Token)), ANY(str), None, None, "p",False).thenRaise(TypeError)
		service.create_service(token,long_name, None, None, "p", False)

Is it normal if this error occur?

  File "/Users/.venv/lib/python3.9/site-packages/mockito/invocation.py", line 410, in raise_
    raise exception
TypeError

[kaste]

Well now you get the TypeError you want. However, I think the testing strategy should be different. We should focus on the function you're actually testing first.

Maybe it looks like:

def make_service(token, service):
   	try:
		return service.create_service(token, ...)
	except TypeError:
		return None

If this is the function under test, then one test could look like:

def test_returns_none_if_create_service_fails(self):
	when(service).create_service(...).thenRaise(TypeError)
 	assert make_service(mock(), service) is None

def test_passes_token_down(self):
	token = mock()
	expect(service).create_service(token, ...)
	make_service(token, service)
	verify(service).create_service(token, ...)

These are just examples. To help you out, you should focus and maybe post the function-under-test. This function uses a dependency (service) or a resource, or does a unwanted side-effect we want to capture. (Anything that is not a pure computation. Something that is not a mathematical function probably.)

Everything else, we also have arg_that, which lets you check an argument basically to program your own matcher, e.g.:

def check_token(token):
	if not isinstance(token.access_token, int): 
		return False
	return True
when(service).create_service(arg_that(check_token), ...).thenRaise(TypeError)
service.create_service(_some_invalid_token_)  # this should throw now

[dvu4]

The function I test is create_service in service.py which importing some external dependencies and may cause some unwanted side-effect

import  exclude_dict, service_names, call_api, 
from .models import AK,  SDetails, Token

class ServiceExists(Exception):
    pass

def create_service(
    token: Token,
    long_name: str,
    dn: Optional[str] = None,
    sn: Optional[str] = None,
    e: Optional[str] = None,
    t: Optional[str] = "p",
    d: bool = False
) -> Optional[AK]:

    if dn is None and (sn is None or e is None):
        raise ValueError(
            "You must set the `dn` or you must set both `sn and e`.")
    
    sp_list = service_names(token)

    if display_name in sp_list:
        raise ServiceExists(f"Service  {dn} exists, please create different name")


    sp_details = SDetails(
        dN=dn,
        tR=f"{long_name}",
        cS="D",
    )

    data = [exclude_dict(sp_details)]
    response = call_api(token, "post", "./service", json=data)
    result = parse_obj_as(AK, response)

    return result

I need to test all the valid inputs, output type is AK , all raise exception and correct message is showed if exception is raised and maybe call_api to see if it executed successfully or not.

[kaste]

Ok, when create_service is the system-under-test then you don't mock it out. You mock its dependencies. Of course create_service is already a complex function and it already has been written before the tests so this is a bit painful.

You look at the dependencies: call_api, service_names, parse_obj_as, exclude_dict. Typically some of them are pure (maybe: parse_obj_as) and could run in a test and some shouldn't really run in a test scenario (probably: call_api). Start with mocking the latter.

In test_create_service.py:

import service as sut   # sut = system-under-test
def test_happy_path(self):
    token, response, result = mock(), mock(), mock()
    when(sut).service_names(token).thenReturn([])
    when(sut).call_api(token, ...).thenReturn(response)   
    when(sut).parse_obj_as(sut.AK, response).thenReturn(result)

    assert sut.create_service(token, "long_name", "dn") == result

Make this test somehow pass.

It gets easier if the first happy path passes.

def test_no_dn_throws(self):
    token, response, result = mock(), mock(), mock()
    when(sut).service_names(token).thenReturn([])
    when(sut).call_api(token, ...).thenReturn(response)   
    when(sut).parse_obj_as(sut.AK, response).thenReturn(result)

    with self.assertRaises(ValueError):
        sut.create_service(token, "long_name")

And from there you test permutations and focus on edge cases and failing inputs. (There is probably already an error in the code as setting dn, sn, and e should throw.)

Does this gets you started?

[dvu4]

I got the error when running test_happy_path(self). Do I need to mock the SDetails and exclude_dict since the output of them will be used as json in call_api ?

Called but not expected:

    call_api(<Dummy id=34323>, 'post', /services', json=[{'app_Other_Type': 'long_name', 'calling_Service': 'DA', 'dN': 'dn'], debug=False)

Stubbed invocations are:

    call_api(<Dummy id=34323>, 'post', '/services', debug=False)

[kaste]

I thought SDetails and exclude_dict are "pure" and you can just fill out the json value manually. Isn't this what the function is about? Taking the arguments, and build a http request from them?

To get you started I actually wrote it with ... (the Ellipsis) -- call_api(token, ...) because in the first test you maybe don't care about the other arguments.

[dvu4]

Thank you for clarifying that for me. I filled out the json value manually and don't care about the arguments (...). I tried to test if the output of service_names is empty, then create a new service .

class AK(BaseModel):
    dM: Any
    code: Optional[int]
    message: str
    status: str

I want to confirm the output of test function, it works.

import service as sut   # sut = system-under-test

def test_happy_path(self):
        when(sut).service_names(...).thenReturn([])
	response = {
		"dM" : "foo",
    		"code" : 200,
    		"message":  "ok",
    		"status" : "200"
    		}

	when(sut).call_api(...).thenReturn(response)
        result = AK(**response)
        when(sut).parse_obj_as(AK, response).thenReturn(result)
	assert sut.create_service("token","long_name", "dn") == result

[dvu4]

I also tried to test if exception is raised, the correct message is showed. It works with pytest. Is there anyway I can do the same in mockito ?

class ServiceExists(Exception):
    Service already exists. Cannot create without the --force flag

import service as sut   # sut = system-under-test
import ServiceExists

def test_exception_message_existed_service(self):
        with pytest.raises(ServiceExists) as err:
	        with mock.patch('sut.service_names') as patched_function:
		        patched_function.return_value = ["ps", "rt", "vc", "ro"]
			sut.create_service("token_foo", "ln_foo", "rt", "sn_foo", "e_foo", "p", False)


	msg = "Service already exists. Cannot create without the --force flag"
	assert msg in str(err.value)
	assert err.type == ServiceExists

[kaste]

I don't see why there should be any difference in the last example:

when(sut).service_names(...).thenReturn(["rt"])
with pytest.raises(ServiceExists) as err:
	sut.create_service("token_foo", "ln_foo", "rt", "sn_foo", "e_foo", "p", False)
# ...

[dvu4]

That works. Thank you !!!