Merge branch 'next' into iss-1798

This commit is contained in:
Pierre-Yves Gérardy 2018-05-04 14:08:39 +02:00 committed by GitHub
commit 4818109c9a
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
107 changed files with 9687 additions and 1550 deletions

21
ospec/LICENSE Normal file
View file

@ -0,0 +1,21 @@
The MIT License (MIT)
Copyright (c) 2017 Leo Horie
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View file

@ -1,15 +1,13 @@
# ospec
ospec [![NPM Version](https://img.shields.io/npm/v/ospec.svg)](https://www.npmjs.com/package/ospec) [![NPM License](https://img.shields.io/npm/l/ospec.svg)](https://www.npmjs.com/package/ospec)
=====
[About](#about) | [Usage](#usage) | [API](#api) | [Goals](#goals)
Noiseless testing framework
Version: 1.2.3
License: MIT
## About
- ~180 LOC
- ~330 LOC including the CLI runner
- terser and faster test code than with mocha, jasmine or tape
- test code reads like bullet points
- assertion code follows [SVO](https://en.wikipedia.org/wiki/Subjectverbobject) structure in present tense for terseness and readability
@ -21,7 +19,7 @@ License: MIT
- `before`/`after`/`beforeEach`/`afterEach` hooks
- test exclusivity (i.e. `.only`)
- async tests and hooks
- explicitly disallows test-space configuration to encourage focus on testing, and to provide uniform test suites across projects
- explicitly regulates test-space configuration to encourage focus on testing, and to provide uniform test suites across projects
## Usage
@ -150,6 +148,22 @@ o("setTimeout calls callback", function(done) {
})
```
Alternativly you can return a promise or even use an async function in tests:
```javascript
o("promise test", function() {
return new Promise(function(resolve) {
setTimeout(resolve, 10)
})
})
```
```javascript
o("promise test", async function() {
await someOtherAsyncFunction()
})
```
By default, asynchronous tests time out after 20ms. This can be changed on a per-test basis using the `timeout` argument:
```javascript
@ -160,7 +174,22 @@ o("setTimeout calls callback", function(done, timeout) {
})
```
Note that the `timeout` function call must be the first statement in its test.
Note that the `timeout` function call must be the first statement in its test. This currently does not work for promise tests. You can combine both methods to do this:
```javascript
o("promise test", function(done, timeout) {
timeout(1000)
someOtherAsyncFunctionThatTakes900ms().then(done)
})
```
```javascript
o("promise test", async function(done, timeout) {
timeout(1000)
await someOtherAsyncFunctionThatTakes900ms()
done()
})
```
Asynchronous tests generate an assertion that succeeds upon calling `done` or fails on timeout with the error message `async test timed out`.
@ -252,7 +281,7 @@ o.run()
The `o.new()` method can be used to create new instances of ospec, which can be run in parallel. Note that each instance will report independently, and there's no aggregation of results.
```javascript
var _o = o.new()
var _o = o.new('optional name')
_o("a test", function() {
_o(1).equals(1)
})
@ -282,7 +311,9 @@ ospec will automatically evaluate all `*.js` files in any folder named `/tests`.
Ospec doesn't work when installed globally. Using global scripts is generally a bad idea since you can end up with different, incompatible versions of the same package installed locally and globally.
To work around this limitation, you can use [`npm-run`](https://www.npmjs.com/package/npm-run) which enables one to run the binaries of locally installed packages.
If you're using a recent version of npm (v5+), you can use run `npx ospec` from your project folder.
Otherwise, to work around this limitation, you can use [`npm-run`](https://www.npmjs.com/package/npm-run) which enables one to run the binaries of locally installed packages.
```
npm install npm-run -g
@ -408,9 +439,24 @@ The arguments that were passed to the function in the last time it was called
---
### void o.run()
### void o.run([Function reporter])
Runs the test suite
Runs the test suite. By default passing test results are printed using
`console.log` and failing test results are printed using `console.error`.
If you have custom continuous integration needs then you can use a
reporter to process [test result data](#result-data) yourself.
If running in Node.js, ospec will call `process.exit` after reporting
results by default. If you specify a reporter, ospec will not do this
and allow your reporter to respond to results in its own way.
---
### Number o.report(results)
The default reporter used by `o.run()` when none are provided. Returns the number of failures, doesn't exit Node.js by itself. It expects an array of [test result data](#result-data) as argument.
---
@ -426,6 +472,74 @@ $o("a test", function() {
$o.run()
```
---
## Result data
Test results are available by reference for integration purposes. You
can use custom reporters in `o.run()` to process these results.
```javascript
o.run(function(results) {
// results is an array
results.forEach(function(result) {
// ...
})
})
```
---
### Boolean result.pass
True if the test passed. **No other keys will exist on the result if this value is true.**
---
### Error result.error
The `Error` object explaining the reason behind a failure.
---
### String result.message
If an exception was thrown inside the corresponding test, this will equal that Error's `message`. Otherwise, this will be a preformatted message in [SVO form](https://en.wikipedia.org/wiki/Subject%E2%80%93verb%E2%80%93object). More specifically, `${subject}\n${verb}\n${object}`.
As an example, the following test's result message will be `"false\nshould equal\ntrue"`.
```javascript
o.spec("message", function() {
o(false).equals(true)
})
```
If you specify an assertion description, that description will appear two lines above the subject.
```javascript
o.spec("message", function() {
o(false).equals(true)("Candyland") // result.message === "Candyland\n\nfalse\nshould equal\ntrue"
})
```
---
### String result.context
A `>`-separated string showing the structure of the test specification.
In the below example, `result.context` would be `testing > rocks`.
```javascript
o.spec("testing", function() {
o.spec("rocks", function() {
o(false).equals(true)
})
})
```
---
## Goals
@ -433,8 +547,8 @@ $o.run()
- Do the most common things that the mocha/chai/sinon triad does without having to install 3 different libraries and several dozen dependencies
- Disallow configuration in test-space:
- Disallow ability to pick between API styles (BDD/TDD/Qunit, assert/should/expect, etc)
- Disallow ability to pick between different reporters
- Disallow ability to add custom assertion types
- Provide a default simple reporter
- Make assertion code terse, readable and self-descriptive
- Have as few assertion types as possible for a workable usage pattern

View file

@ -17,6 +17,7 @@ function traverseDirectory(pathname, callback) {
var promises = []
for (var i = 0; i < pathnames.length; i++) {
if (pathnames[i] === "node_modules") continue
if (pathnames[i][0] === ".") continue
pathnames[i] = path.join(pathname, pathnames[i])
promises.push(traverseDirectory(pathnames[i], callback))
}

View file

@ -1,9 +1,16 @@
/* eslint-disable no-bitwise, no-process-exit */
/* eslint-disable global-require, no-bitwise, no-process-exit */
"use strict"
module.exports = new function init() {
;(function(m) {
if (typeof module !== "undefined") module["exports"] = m()
else window.o = m()
})(function init(name) {
var spec = {}, subjects = [], results, only = null, ctx = spec, start, stack = 0, nextTickish, hasProcess = typeof process === "object", hasOwn = ({}).hasOwnProperty
if (name != null) spec[name] = ctx = {}
try {throw new Error} catch (e) {
var ospecFileName = e.stack && (/[\/\\](.*?):\d+:\d+/).test(e.stack) ? e.stack.match(/[\/\\](.*?):\d+:\d+/)[1] : null
}
function o(subject, predicate) {
if (predicate === undefined) {
if (results == null) throw new Error("Assertions should not occur outside test definitions")
@ -47,10 +54,35 @@ module.exports = new function init() {
spy.callCount = 0
return spy
}
o.run = function() {
o.cleanStackTrace = function(error) {
// For IE 10+ in quirks mode, and IE 9- in any mode, errors don't have a stack
if (error.stack == null) return ""
var i = 0, header = error.message ? error.name + ": " + error.message : error.name, stack
// some environments add the name and message to the stack trace
if (error.stack.indexOf(header) === 0) {
stack = error.stack.slice(header.length).split(/\r?\n/)
stack.shift() // drop the initial empty string
} else {
stack = error.stack.split(/\r?\n/)
}
if (ospecFileName == null) return stack.join("\n")
// skip ospec-related entries on the stack
while (stack[i].indexOf(ospecFileName) !== -1) i++
// now we're in user code
return stack[i]
}
o.run = function(reporter) {
results = []
start = new Date
test(spec, [], [], report)
test(spec, [], [], function() {
setTimeout(function () {
if (typeof reporter === "function") reporter(results)
else {
var errCount = o.report(results)
if (hasProcess && errCount !== 0) process.exit(1)
}
})
})
function test(spec, pre, post, finalize) {
pre = [].concat(pre, spec["__beforeEach"] || [])
@ -82,41 +114,56 @@ module.exports = new function init() {
if (cursor === fns.length) return
var fn = fns[cursor++]
var timeout = 0, delay = 200, s = new Date
var isDone = false
function done(err) {
if (err) {
if (err instanceof Error) record(err.message, err)
else record(String(err))
subjects.pop()
next()
}
if (timeout !== undefined) {
timeout = clearTimeout(timeout)
if (delay !== Infinity) record(null)
if (!isDone) next()
else throw new Error("`" + arg + "()` should only be called once")
isDone = true
}
else console.log("# elapsed: " + Math.round(new Date - s) + "ms, expected under " + delay + "ms")
}
function startTimer() {
timeout = setTimeout(function() {
timeout = undefined
record("async test timed out")
next()
}, Math.min(delay, 2147483647))
}
if (fn.length > 0) {
var timeout = 0, delay = 200, s = new Date
var isDone = false
var body = fn.toString()
var arg = (body.match(/\(([\w$]+)/) || body.match(/([\w$]+)\s*=>/) || []).pop()
if (body.indexOf(arg) === body.lastIndexOf(arg)) throw new Error("`" + arg + "()` should be called at least once")
try {
fn(function done() {
if (timeout !== undefined) {
timeout = clearTimeout(timeout)
if (delay !== Infinity) record(null)
if (!isDone) next()
else throw new Error("`" + arg + "()` should only be called once")
isDone = true
}
else console.log("# elapsed: " + Math.round(new Date - s) + "ms, expected under " + delay + "ms")
}, function(t) {delay = t})
fn(done, function(t) {delay = t})
}
catch (e) {
if (e instanceof Error) record(e.message, e)
else record(String(e))
subjects.pop()
next()
done(e)
}
if (timeout === 0) {
timeout = setTimeout(function() {
timeout = undefined
record("async test timed out")
next()
}, Math.min(delay, 2147483647))
startTimer()
}
}
else {
fn()
nextTickish(next)
var p = fn()
if (p && p.then) {
startTimer()
p.then(function() { done() }, done)
} else {
nextTickish(next)
}
}
}
}
@ -200,11 +247,13 @@ module.exports = new function init() {
}
result.context = subjects.join(" > ")
result.message = message
result.error = error.stack
result.error = error
}
results.push(result)
}
function serialize(value) {
if (hasProcess) return require("util").inspect(value)
if (value === null || (typeof value === "object" && !(value instanceof Array)) || typeof value === "number") return String(value)
else if (typeof value === "function") return value.name || "<anonymous function>"
try {return JSON.stringify(value)} catch (e) {return String(value)}
@ -213,23 +262,24 @@ module.exports = new function init() {
return hasProcess ? "\x1b[31m" + message + "\x1b[0m" : "%c" + message + "%c "
}
function report() {
var status = 0
o.report = function (results) {
var errCount = 0
for (var i = 0, r; r = results[i]; i++) {
if (!r.pass) {
var stackTrace = r.error.match(/^(?:(?!Error|[\/\\]ospec[\/\\]ospec\.js).)*$/m)
var stackTrace = o.cleanStackTrace(r.error)
console.error(r.context + ":\n" + highlight(r.message) + (stackTrace ? "\n\n" + stackTrace + "\n\n" : ""), hasProcess ? "" : "color:red", hasProcess ? "" : "color:black")
status = 1
errCount++
}
}
console.log(
(name ? name + ": " : "") +
results.length + " assertions completed in " + Math.round(new Date - start) + "ms, " +
"of which " + results.filter(function(result){return result.error}).length + " failed"
)
if (hasProcess && status === 1) process.exit(1)
return errCount
}
if(hasProcess) {
if (hasProcess) {
nextTickish = process.nextTick
} else {
nextTickish = function fakeFastNextTick(next) {
@ -239,4 +289,4 @@ module.exports = new function init() {
}
return o
}
})

View file

@ -1,6 +1,6 @@
{
"name": "ospec",
"version": "1.2.3",
"version": "1.4.0",
"description": "Noiseless testing framework",
"main": "ospec.js",
"directories": {
@ -12,5 +12,5 @@
"bin": {
"ospec": "./bin/ospec"
},
"repository": "lhorie/mithril.js#rewrite"
"repository": "MithrilJS/mithril.js"
}

View file

@ -18,6 +18,77 @@ new function(o) {
o.run()
}(o)
new function(o) {
var clone = o.new()
clone.spec("clone", function() {
clone("fail", function() {
clone(true).equals(false)
})
clone("pass", function() {
clone(true).equals(true)
})
})
// Predicate test passing on clone results
o.spec("reporting", function() {
o("reports per instance", function(done, timeout) {
timeout(100) // Waiting on clone
clone.run(function(results) {
o(typeof results).equals("object")
o("length" in results).equals(true)
o(results.length).equals(2)("Two results")
o("error" in results[0] && "pass" in results[0]).equals(true)("error and pass keys present in failing result")
o(!("error" in results[1]) && "pass" in results[1]).equals(true)("only pass key present in passing result")
o(results[0].pass).equals(false)("Test meant to fail has failed")
o(results[1].pass).equals(true)("Test meant to pass has passed")
done()
})
})
o("o.report() returns the number of failures", function () {
var log = console.log, error = console.error
console.log = o.spy()
console.error = o.spy()
function makeError(msg) {try{throw msg ? new Error(msg) : new Error} catch(e){return e}}
try {
var errCount = o.report([{pass: true}, {pass: true}])
o(errCount).equals(0)
o(console.log.callCount).equals(1)
o(console.error.callCount).equals(0)
errCount = o.report([
{pass: false, error: makeError("hey"), message: "hey"}
])
o(errCount).equals(1)
o(console.log.callCount).equals(2)
o(console.error.callCount).equals(1)
errCount = o.report([
{pass: false, error: makeError("hey"), message: "hey"},
{pass: true},
{pass: false, error: makeError("ho"), message: "ho"}
])
o(errCount).equals(2)
o(console.log.callCount).equals(3)
o(console.error.callCount).equals(3)
} catch (e) {
o(1).equals(0)("Error while testing the reporter")
}
console.log = log
console.error = error
})
})
}(o)
o.spec("ospec", function() {
o.spec("sync", function() {
var a = 0, b = 0, illegalAssertionThrows = false
@ -107,7 +178,7 @@ o.spec("ospec", function() {
o(output).deepEquals({tag: "div", children: children})
})
})
o.spec("async", function() {
o.spec("async callback", function() {
var a = 0, b = 0
o.before(function(done) {
@ -148,4 +219,63 @@ o.spec("ospec", function() {
})
})
})
o.spec("stack trace cleaner", function() {
o("handles line breaks", function() {
try {
throw new Error("line\nbreak")
} catch(error) {
var trace = o.cleanStackTrace(error)
o(trace).notEquals("break")
o(trace.includes("test-ospec.js")).equals(true)
}
})
})
o.spec("async promise", function() {
var a = 0, b = 0
function wrapPromise(fn) {
return new Promise((resolve, reject) => {
callAsync(() => {
try {
fn()
resolve()
} catch(e) {
reject(e)
}
})
})
}
o.before(function() {
return wrapPromise(() => {
a = 1
})
})
o.after(function() {
return wrapPromise(function() {
a = 0
})
})
o.beforeEach(function() {
return wrapPromise(function() {
b = 1
})
})
o.afterEach(function() {
return wrapPromise(function() {
b = 0
})
})
o("promise functions", function() {
return wrapPromise(function() {
o(a).equals(b)
o(a).equals(1)("a and b should be initialized")
})
})
})
})