Ability to inject middleware into rule

The builder for rule could be modified so you can pass custom middleware into Hoverfly

While injecting middleware is not yet available for hoverfy-java, one work around would be starting up hoverfly with middleware outside JVM, eg. :

hoverctl middleware --binary python --script middleware.py

Then use the external hoverfly in hoverfly-java by setting the useRemoteInstance option:

@ClassRule
    public static HoverflyRule hoverflyRule = HoverflyRule.inSimulationMode(classpath("test-service.json"),
            configs().useRemoteInstance("localhost"));

@tommysitu

@tommysitu, we will give this work around a go. Thanks!

@tommysitu

@tommysitu we are able to invoke the middleware with the above workaround in a JUnit test!

The next progression would be to move this invocation code into a networkHelper class that our app can call and get the synthesized data back. The only caveat is the HoverflyConfig and SimulationSource are both in the io.specto.hoverfly.junit.core package.

We like to have a method like this...

public static String getRoomScheduleSimulation() {

    try {
        HoverflyRule hoverflyRule = HoverflyRule.inCaptureOrSimulationMode(("javaSimulation.json"), HoverflyConfig.configs().useRemoteInstance());

        Process p = Runtime.getRuntime().exec("curl --insecure --proxy http://localhost:8500curl --insecure --proxy http://localhost:8500 https://wpa.steelcase.com/rooms/a7155e79-842e-11e6-a1a5-000d3a321a2e/bookings");

        p.waitFor();
        BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream()));

        String line = "";
        StringBuffer output = new StringBuffer();

        while((line = reader.readLine()) != null) {
            output.append(line + "\n");

        }

        String wholeResponseString = output.toString();

//or parse the string into Model here...

        return wholeResponseString;

    } catch (Exception e) {
        e.printStackTrace();
    }
    return null;
}

Any recommendations/thoughts?

Thanks!

@jxlampdx

@jxlampdx I am trying to understand your problem a bit better. Do you want to simulate a Restful endpoint (https://wpa.steelcase.com/rooms/a7155e79-842e-11e6-a1a5-000d3a321a2e/bookings) in your JUnit test?

Also I am assuming that you are trying to test your application that depends on the synthesised response from that endpoint?

@tommysitu

@tommysitu , yes and yes. I like to be able to simulate and return a middleware injected response from Junit test and also able to have a helper method so that we can call it to get the synthesized data and populate the views...

The problem we are trying to solve is to synthesize response from services that are not yet available from our backend.

looking for a Gradle Build that is a Compile if it is available? Currently we have testCompile 'io.specto:hoverfly-java:0.3.8'

@jxlampdx

@jxlampdx It sounds like you want to use the simulation for both your JUnit test, and also inside your application for the time being.

For JUnit test, you only need to add the JUnit Rule to your test class:

    @ClassRule
    public static HoverflyRule hoverflyRule = HoverflyRule.inSimulationMode(classpath("javaSimulation.json"),
            HoverflyConfig.configs().useRemoteInstance());

The JUnit rule would auto-configure JVM proxy and default SSL, therefore you don't need these flags "--insecure --proxy http://localhost:8500". Any particular reason you are invoking curl in java code?

To use the simulation outside JUnit test, for example embedded in your application to be a stub server, you can invoke this when your application starting up:

try(Hoverfly hoverfly = new Hoverfly(HoverflyConfig.configs().useRemoteInstance(), HoverflyMode.SIMULATE)) {

            hoverfly.importSimulation(SimulationSource.classpath("javaSimulation.com"));
            hoverfly.start();
        }

Yes, you can get hoverfly-java with compile / testCompile in Gradle.

For hoverfly-java to work with both compile/testcompile. I changed the build.graddle to

compile 'io.specto:hoverfly-java:0.3.8 instead of the testCompile 'io.specto:hoverfly-java:0.3.8' .

is that correct?

this is my dependencies ...

dependencies {
compile fileTree(dir: 'libs', include: ['*.jar'])
androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', {
exclude group: 'com.android.support', module: 'support-annotations'
})

compile 'io.specto:hoverfly-java:0.3.8'

compile 'com.android.support:appcompat-v7:25.1.0'
compile 'com.android.support:support-v4:25.1.0'
compile 'com.android.support:design:25.1.0'


testCompile 'junit:junit:4.12'

}

and getting this error...

FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':app:transformResourcesWithMergeJavaResForDebug'.

com.android.build.api.transform.TransformException: com.android.builder.packaging.DuplicateFileException: Duplicate files copied in APK META-INF/LICENSE
File1: /Users/jlam1/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.8.4/de3570327cf8d1d4f37920535c51a1f74225a6de/jackson-annotations-2.8.4.jar
File2: /Users/jlam1/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.jaxrs/jackson-jaxrs-base/2.8.4/6c0ceb3c9fed2e225b0cc2a45533574df393f606/jackson-jaxrs-base-2.8.4.jar
File3: /Users/jlam1/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-databind/2.8.5/b3035f37e674c04dafe36a660c3815cc59f764e2/jackson-databind-2.8.5.jar
File4: /Users/jlam1/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.jaxrs/jackson-jaxrs-json-provider/2.8.4/839366ece31829a19cb15719b2b54a3f9f91148d/jackson-jaxrs-json-provider-2.8.4.jar
File5: /Users/jlam1/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.module/jackson-module-jaxb-annotations/2.8.4/d2eec7cf6c4284f7d5f0b1a72dc7cfa9d6bb579d/jackson-module-jaxb-annotations-2.8.4.jar
File6: /Users/jlam1/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.8.5/60d059f5d2930ccd1ef03535b713fd9f933d1ba7/jackson-core-2.8.5.jar

Using compile is fine, it looks like you need to exclude Jackson-core dependency from hoverfly-java.

I excluded t 'META-INF/jersey-module-version' and gotten passed the above. but now gotten another runtime error. Tried adding the

compileOptions {
sourceCompatibility JavaVersion.VERSION_1_7
targetCompatibility JavaVersion.VERSION_1_7
}
and also adding
multiDexEnabled true

clean and rebuilt but could get pass this error below. I also tried to remove all the dependencies and added them back one at a time, once I added back

compile 'io.specto:hoverfly-java:0.3.8', will encounter the runtime error above.

Any other ideas? Thanks!

:app:transformClassesWithDexForDebug
AGPBI: {"kind":"error","text":"Error converting bytecode to dex:\nCause: Dex cannot parse version 52 byte code.\nThis is caused by library dependencies that have been compiled using Java 8 or above.\nIf you are using the \u0027java\u0027 gradle plugin in a library submodule add \ntargetCompatibility \u003d \u00271.7\u0027\nsourceCompatibility \u003d \u00271.7\u0027\nto that submodule\u0027s build.gradle file.","sources":[{}],"original":"UNEXPECTED TOP-LEVEL EXCEPTION:\njava.lang.RuntimeException: Exception parsing classes\n\tat com.android.dx.command.dexer.Main.processClass(Main.java:775)\n\tat com.android.dx.command.dexer.Main.processFileBytes(Main.java:741)\n\tat com.android.dx.command.dexer.Main.access$1200(Main.java:88)\n\tat com.android.dx.command.dexer.Main$FileBytesConsumer.processFileBytes(Main.java:1683)\n\tat com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:284)\n\tat com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:166)\n\tat com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:144)\n\tat com.android.dx.command.dexer.Main.processOne(Main.java:695)\n\tat com.android.dx.command.dexer.Main.processAllFiles(Main.java:592)\n\tat com.android.dx.command.dexer.Main.runMultiDex(Main.java:376)\n\tat com.android.dx.command.dexer.Main.run(Main.java:290)\n\tat com.android.builder.internal.compiler.DexWrapper.run(DexWrapper.java:54)\n\tat com.android.builder.core.DexByteCodeConverter.lambda$dexInProcess$0(DexByteCodeConverter.java:173)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\n\tat java.lang.Thread.run(Thread.java:745)\nCaused by: com.android.dx.cf.iface.ParseException: bad class file magic (cafebabe) or version (0034.0000)\n\tat com.android.dx.cf.direct.DirectClassFile.parse0(DirectClassFile.java:476)\n\tat com.android.dx.cf.direct.DirectClassFile.parse(DirectClassFile.java:406)\n\tat com.android.dx.cf.direct.DirectClassFile.parseToInterfacesIfNecessary(DirectClassFile.java:388)\n\tat com.android.dx.cf.direct.DirectClassFile.getMagic(DirectClassFile.java:251)\n\tat com.android.dx.command.dexer.Main.parseClass(Main.java:787)\n\tat com.android.dx.command.dexer.Main.access$1600(Main.java:88)\n\tat com.android.dx.command.dexer.Main$ClassParserTask.call(Main.java:1722)\n\tat com.android.dx.command.dexer.Main.processClass(Main.java:773)\n\t... 16 more\n","tool":"Dex"}
AGPBI: {"kind":"error","text":"1 error; aborting","sources":[{}]}

FAILED

FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':app:transformClassesWithDexForDebug'.

com.android.build.api.transform.TransformException: java.lang.RuntimeException: java.lang.RuntimeException: com.android.ide.common.process.ProcessException: java.util.concurrent.ExecutionException: com.android.ide.common.process.ProcessException: Return code 1 for dex process

That's because hoverfly-java requires java 8 to compile.

@tommysitu

I see. now that I stepped back and look, if we have the remote instance running as a proxy, we don't actually need the integration in the Android app to invoke it. As long as we inject the middleware code before we call the service in the Android app, we should be good.

Thanks! @tommysitu

@jxlampdx

No problem @jxlampdx. We are always looking for interesting use cases of Hoverfly. Could you be contacted via email? We are quite keen to find out how you are using hoverfly for mobile development. Thanks.

If it helps we have a contact form near the foot of the http://specto.io/ page so no one needs to disclose their email here.

@JohnFDavenport

@JohnFDavenport and @tommysitu, I filled out the contact form. Thanks again

@mogronalol

Hey,

I have been thinking about middleware a lot recently and how it should be used.

There are currently two ways of executing middleware in Hoverfly. The first way is through a process that is started for each request. This has performance issues that occur as you start increase the load of Hoverfly. The reason for this is due to spinning up a new process for each request in flight. The result can be hundreds of child processes all spawned from Hoverfly.

The other way, (which is currently undocumented as of the time of writing this) is running middleware over HTTP. This allows you to turn your middleware into a HTTP server (or maybe a serverless function a la AWS Lambda). Using this approach means that you generally only need one process to run the HTTP server which can be reused by Hoverfly, reducing the performance overhead with process creation.

I have been thinking for a while that maybe the default way people should be implementing middleware is as HTTP servers. I believe its simpler and it is definitely more performant.

With the middleware context out of the way, I think implementing middleware as a HTTP server in hoverfly-java could be much easier than worrying about middleware processes. In golang, it is very easy to produce HttpHandler functions that match a HTTPHandler interface to quickly produce HTTP servers. Maybe hoverfly-java users could define their middleware in their tests as a MiddlewareFunction that is then passed to hoverfly-java which then serves it and provides the address to the running instance of Hoverfly?

Your opinions @mogronalol and @tommysitu?

@benjih

@benjih Yes I'm in agreement. We can implement a webserver middleware which starts in the same process as the test, as part of the Java code. This actually frees Java developers up to do whatever they want with the data.

@benjih

Seems like a great idea @benjih @mogronalol but I have some questions. (1) Would this be generic enough to be used in other languages? (2) Throttling: You could still flood the webserver with thousands of requests during a performance test. At the moment responsibility is being passed from the very efficient multi-threading Go process to a poorly scaling Linux process. The issues start when the tester wants to do something long-running like add a delay. How much better will this solution be? A webserver is unlikely to be more efficient than Go and will hit the same issues, albeit at a higher load. If the webserver calls out to a database will it have to manage a connection pool? It may be better to throttle at the Hoverfly end of things using some form of internal queue to maintain order. That might allow us to keep the webserver relatively simple. (3) When. After Hoverfly 1.0 please.

This example would be webserver in Java as part of the same code / process as the test
We already support webserver mode in Hoverfly. This is just a suggestion on how it could be used with Hoverfly Java. It's already faster than binary middleware because you only have to start the process once. Also, contrary to popular belief the JVM is still very fast and scalable, even when doing things in parallel. How throttling is implemented is up to the developer. Hoverfly already supports delays, as I'd probably just use that.

@JohnFDavenport

@JohnFDavenport It won't be generic enough for other languages. We will need to start up embedded web server in JUnit tests using lightweight web framework such as Spark: https://github.com/perwendel/spark/

For simulating delay and network latency, it is better to have native support from hoverfly, and avoid the use of middleware whenever we could.

@dipak-pawar

Support for using local middleware is available, thanks to @dipak-pawar!

Navigation Menu

Repository navigation

Ability to inject middleware into rule #7

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Ability to inject middleware into rule #7

Description

Activity

tommysitu commented on Feb 24, 2017

jxlampdx commented on Feb 24, 2017

jxlampdx commented on Feb 24, 2017

tommysitu commented on Feb 24, 2017

jxlampdx commented on Feb 24, 2017

jxlampdx commented on Feb 24, 2017

tommysitu commented on Feb 24, 2017

tommysitu commented on Feb 24, 2017

jxlampdx commented on Feb 24, 2017

jxlampdx commented on Feb 24, 2017

tommysitu commented on Feb 25, 2017

jxlampdx commented on Feb 27, 2017

tommysitu commented on Feb 27, 2017

jxlampdx commented on Feb 27, 2017

tommysitu commented on Feb 28, 2017

JohnFDavenport commented on Feb 28, 2017

jxlampdx commented on Feb 28, 2017

benjih commented on Mar 15, 2017

mogronalol commented on Mar 16, 2017

JohnFDavenport commented on Mar 20, 2017

mogronalol commented on Mar 20, 2017

tommysitu commented on Mar 21, 2017

tommysitu commented on Jan 29, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions