๐งฉ Plugins
+ +Plugins lets add new features with additional content to rendered metrics and are coded with [JavaScript modules](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules). + +New plugins are welcomed, but maintainers have no obligation to maintain them. +It means you (as author) may be notified about open issues regarding related plugin. + +
+
+```
+
+It will create a new folder in [`source/plugins`](https://github.com/lowlighter/metrics/tree/master/source/plugins) with the following files:
+- A `README.md` to describe your plugin and document it
+- An `index.mjs` with minimal plugin code
+- A `metadata.yml` which list plugin attributes and inputs
+- A `tests.yml` for unit tests
+
+Here are some guidelines to follow about plugins:
+- They should never be dependent on output produced by other plugins (though allowed to re-use core and base data)
+ - It allows parallelization of plugins execution
+ - It avoids creating inter-dependencies which makes it confusing for both developers and users
+- Use of new external dependencies should be avoided
+ - Adding new libraries to use only ~5% of its possibilities is just a waste
+ - For APIs, most of the time a few HTTP calls instead of installing a full SDK wrapper is more than sufficient
+ - `imports` probably already contains a library or a function that can help you achieving what you want
+ - It also add more unstability as it external changes are
+- Use of raw commands should be avoided when (spawning sub-process)
+ - It allows metrics to be platform agnostic (i.e. working on most OS)
+ - If mandatory:
+ - Use [`which`](https://linux.die.net/man/1/which) detect whether command is available
+ - For Windows, wrap command with [WSL](https://docs.microsoft.com/windows/wsl/about)
+- Errors should be handled gracefully with error messages
+- Plugins arguments should **NEVER** be directly edited from inside a plugin
+ - These are used by all plugins, including core and base so it would create unattended side effects
+- They should let end user with some customization options (limit entries, detailed output, etc.)
+
+You'll also need to an unused [emoji](https://emojipedia.org) to use as your plugin icon.
+
+Plugins are auto-loaded based on their folder existence, so there's no need to register them somewhere.
+
+
+
+๐ฌ Creating a new plugin from scratch
+ +Find a cool name for your plugin and run: +```shell +npm run quickstart -- plugin
+๐ฌ Implementing
+
+Default exported function in `index.mjs` will receive the following inputs:
+- `login`, set to GitHub login
+- `q`, with query parameters (formatted with dots (`.`) instead of underscores (`_`) and without `plugin_` prefix)
+- `imports`, with libraries and utilitaries
+ - `imports.url` for [NodeJS `url` library](https://nodejs.org/api/url.html)
+ - `imports.os` for [NodeJS `os` library](https://nodejs.org/api/os.html)
+ - `imports.fs` for [NodeJS `fs` library](https://nodejs.org/api/fs.html)
+ - `imports.paths` for [NodeJS `paths` library](https://nodejs.org/api/paths.html)
+ - `imports.util` for [NodeJS `util` library](https://nodejs.org/api/util.html)
+ - `imports.imgb64` for [renanbastos93/image-to-base64](https://github.com/renanbastos93/image-to-base64)
+ - `imports.axios` for [axios/axios](https://github.com/axios/axios)
+ - `imports.puppeteer` for [puppeteer/puppeteer](https://github.com/puppeteer/puppeteer)
+ - `imports.run` is an helper to run raw command
+ - `imports.shuffle` is an helper to shuffle array
+ - `imports.__module` is an helper to find `__dirname` from a module `import.meta.url`
+ - And more...
+- `data` and `computed`, with all data gathered from core and base
+- `graphql` and `rest`, with authenticated [octokit clients](https://github.com/octokit) (for GitHub APIs)
+- `queries`, with autoloaded GraphQL queries and replacers
+- `account`, set to account type ("user" or "organization")
+
+Second input contains configuration settings from [settings.json](https://github.com/lowlighter/metrics/blob/master/settings.example.json) (which is mostly used by web instances) and all also user inputs of type `token`.
+
+As said previously, plugins arguments should **NEVER** be directly edited from it, since these are used by all plugins, including core and base so it would create unattended side effects.
+
+As for data gathering:
+ - Related to GitHub, use `graphql` (for [GraphQL API](https://docs.github.com/en/graphql)) or `rest` [REST API](https://docs.github.com/en/rest)
+ - From Third-Party services, use [`imports.axios`](https://github.com/axios/axios) to make APIs calls
+ - In last resort, use `imports.puppeteer`
+
+For GraphQL queries, use `queries` which will auto-load all queries from `queries` directory and will lets you create custom queries on the fly.
+
+For example:
+```js
+//Calling this
+ await graphql(queries.myquery({login:"github-user", account:"user"}))
+
+//With this in source/queries/myquery.graphql
+ query MyQuery {
+ $account(login: "$login") {
+ name
+ }
+ }
+
+//Will have the same result as calling this
+ await graphql(`
+ query MyQuery {
+ user(login: "github-user") {
+ name
+ }
+ }
+ `)
+```
+
+
+
+
+๐ฌ Implementing index.mjs and gathering new data from external APIs
+
+Default exported function in `index.mjs` will receive the following inputs:
+- `login`, set to GitHub login
+- `q`, with query parameters (formatted with dots (`.`) instead of underscores (`_`) and without `plugin_` prefix)
+- `imports`, with libraries and utilitaries
+ - `imports.url` for [NodeJS `url` library](https://nodejs.org/api/url.html)
+ - `imports.os` for [NodeJS `os` library](https://nodejs.org/api/os.html)
+ - `imports.fs` for [NodeJS `fs` library](https://nodejs.org/api/fs.html)
+ - `imports.paths` for [NodeJS `paths` library](https://nodejs.org/api/paths.html)
+ - `imports.util` for [NodeJS `util` library](https://nodejs.org/api/util.html)
+ - `imports.imgb64` for [renanbastos93/image-to-base64](https://github.com/renanbastos93/image-to-base64)
+ - `imports.axios` for [axios/axios](https://github.com/axios/axios)
+ - `imports.puppeteer` for [puppeteer/puppeteer](https://github.com/puppeteer/puppeteer)
+ - `imports.run` is an helper to run raw command
+ - `imports.shuffle` is an helper to shuffle array
+ - `imports.__module` is an helper to find `__dirname` from a module `import.meta.url`
+ - And more...
+- `data` and `computed`, with all data gathered from core and base
+- `graphql` and `rest`, with authenticated [octokit clients](https://github.com/octokit) (for GitHub APIs)
+- `queries`, with autoloaded GraphQL queries and replacers
+- `account`, set to account type ("user" or "organization")
+
+Second input contains configuration settings from [settings.json](https://github.com/lowlighter/metrics/blob/master/settings.example.json) (which is mostly used by web instances) and all also user inputs of type `token`.
+
+As said previously, plugins arguments should **NEVER** be directly edited from it, since these are used by all plugins, including core and base so it would create unattended side effects.
+
+As for data gathering:
+ - Related to GitHub, use `graphql` (for [GraphQL API](https://docs.github.com/en/graphql)) or `rest` [REST API](https://docs.github.com/en/rest)
+ - From Third-Party services, use [`imports.axios`](https://github.com/axios/axios) to make APIs calls
+ - In last resort, use `imports.puppeteer`
+
+For GraphQL queries, use `queries` which will auto-load all queries from `queries` directory and will lets you create custom queries on the fly.
+
+For example:
+```js
+//Calling this
+ await graphql(queries.myquery({login:"github-user", account:"user"}))
+
+//With this in source/queries/myquery.graphql
+ query MyQuery {
+ $account(login: "$login") {
+ name
+ }
+ }
+
+//Will have the same result as calling this
+ await graphql(`
+ query MyQuery {
+ user(login: "github-user") {
+ name
+ }
+ }
+ `)
+```
+
+
+๐ฌ Filling
+
+`metadata.yml` is a mandatory file which describes what inputs are allowed, which entities are supported, etc.
+
+Here's an example:
+```yaml
+name: "๐งฉ Plugin name (with emoji icon)"
+cost: Estimates how many GitHub requests is used during plugin execution ("N/A" for Third-Party services)
+supports:
+ - user # Support users account
+ - organization # Support organizations account
+ - repository # Support repositories metrics
+inputs:
+
+ # A comment detailing input purposes
+ # An input must have at least a "description" and a "default" (used to generated GitHub Action `action.yml`)
+ plugin_input:
+ description: Short description (few words)
+ type: boolean
+ default: no
+```
+
+Because of GitHub Actions limitations, only strings and numbers are actually supported by `action.yml` inputs.
+Metrics apply additional post-processing to handle inputs.
+
+Supported input types are `boolean`, `string`, `number`, `array` and `json`.
+
+- Allowed values for `string` and `array` may be restricted using `values` attribute
+ - Special default values `.user.login`, `.user.twitter` and `.user.website` will respectively be replaced by user's login, Twitter username and website (not available when `token` is set to `NOT_NEEDED` by user )
+- Lower and upper limits for `number` may be set using `min` and `max` attribute
+- Array `format` attribute define how string should be splitted (`comma-separated` or `space-separated`)
+
+You can additionally specify an `example` which will also be used in web instance input placeholder.
+
+Inputs will be available through `imports.metadata.plugins.name.inputs` with correct typing and default values (`plugin_` prefix will be dropped, and all underscored (`_`) will be changed to dots (`.`) instead):
+```javascript
+//Load inputs
+ let {limit, "limit.field":limit_field} = imports.metadata.plugins.name.inputs({data, account, q})
+```
+
+Additionally, if `account` isn't supported, this method will automatically prevent your plugin from running by throwing an error.
+
+
+
+๐ฌ Filling metadata.yml
+
+`metadata.yml` is a mandatory file which describes what inputs are allowed, which entities are supported, etc.
+
+Here's an example:
+```yaml
+name: "๐งฉ Plugin name (with emoji icon)"
+cost: Estimates how many GitHub requests is used during plugin execution ("N/A" for Third-Party services)
+supports:
+ - user # Support users account
+ - organization # Support organizations account
+ - repository # Support repositories metrics
+inputs:
+
+ # A comment detailing input purposes
+ # An input must have at least a "description" and a "default" (used to generated GitHub Action `action.yml`)
+ plugin_input:
+ description: Short description (few words)
+ type: boolean
+ default: no
+```
+
+Because of GitHub Actions limitations, only strings and numbers are actually supported by `action.yml` inputs.
+Metrics apply additional post-processing to handle inputs.
+
+Supported input types are `boolean`, `string`, `number`, `array` and `json`.
+
+- Allowed values for `string` and `array` may be restricted using `values` attribute
+ - Special default values `.user.login`, `.user.twitter` and `.user.website` will respectively be replaced by user's login, Twitter username and website (not available when `token` is set to `NOT_NEEDED` by user )
+- Lower and upper limits for `number` may be set using `min` and `max` attribute
+- Array `format` attribute define how string should be splitted (`comma-separated` or `space-separated`)
+
+You can additionally specify an `example` which will also be used in web instance input placeholder.
+
+Inputs will be available through `imports.metadata.plugins.name.inputs` with correct typing and default values (`plugin_` prefix will be dropped, and all underscored (`_`) will be changed to dots (`.`) instead):
+```javascript
+//Load inputs
+ let {limit, "limit.field":limit_field} = imports.metadata.plugins.name.inputs({data, account, q})
+```
+
+Additionally, if `account` isn't supported, this method will automatically prevent your plugin from running by throwing an error.
+
+
+
+
+<% } %>
+```
+
+- First conditional statement ensures that partial is displayed only when plugin is enabled
+- Nested conditional statement check plugin output
+ - If it failed, an error message instead will be displayed instead
+ - If it succeeded, second section in render.
+
+Additional CSS rules may be added to `style.css` of edited template, but ensure it does not break other plugins rendering.
+
+
+
+
+๐ฌ Creating a new partial
+ +In templates you want to support, create a new `.ejs` file in `partials` folder and paste the following for a quick start: +```html +<% if (plugins./* your plugin name */) { %> +
+ <% if (plugins./* your plugin name */.error) { %>
+
+
+ <% } else { %>
+
+ <%# Do stuff in there -%>
+
+ <% } %>
+
+
+
+ <%= plugins./* your plugin name */.error.message %>
+
+
+
+
+๐ฌ Fast prototyping and testing
+ +The easiest way to test and prototype your plugin is to setup a web instance. See [documentation](https://github.com/lowlighter/metrics#%EF%B8%8F-deploying-your-own-web-instance-15-min-setup-depending-on-your-sysadmin-knowledge) for more informations about that. + +Open a browser and try to generate metrics with new your plugin enabled to see if it works as expected: +``` +http://localhost:3000/your-github-login?base=0&your-plugin-name=1 +``` + +Once ready, define test cases in your plugin directory `tests.yml`. + +These tests will be run with: + - Metrics action + - Metrics web instance + - Metrics web instance placeholder (rendered by browser) + +Most APIs (including GitHub) usually have a rate-limit to ensure quality of service. +This is why APIs output must be mocked and added in [`source/app/mocks/api/`](/source/app/mocks/api) in order for tests to be able to be performed anytime. + +Files from these directories are auto-loaded, so you just need to create new functions (see other mocked data for examples). + +Finally, edit [source/app/web/statics/app.placeholder.js](https://github.com/lowlighter/metrics/blob/master/source/app/web/statics/app.placeholder.js) to add mocked result (but this time from metrics server) so users will be able to render placeholder preview in web instance. + +
+` tag **MUST** remain present (along with `
+
+
+#### โน๏ธ Examples workflows
+
+[โก๏ธ Available options for this plugin](metadata.yml)
+
+'''yaml
+- uses: lowlighter/metrics@latest
+ with:
+ # ... other options
+ plugin_custom: yes
+'''
+
+```
+
+Note that you **must** keep .
+
+
+
+
+## ๐ฌ Behind the scenes
+
+This section explore some topics which explain globally how metrics was designed and how it works.
๐ฌ Submitting a pull request
+ +Ensure that: +- `metadata.yml` is correctly filled +- `tests.yml` has defined test cases +- `mocks/api` has mocked data for external APIs +- `app.placeholder.js` has been updated with mocked plugin output +- `README.md` of plugin explain how plugin works + - `|
+ |
+
๐๏ธ Project structure
@@ -206,433 +522,62 @@ Below is a list of used packages.
-
-```
+Images (and custom fonts) are encoded into base64 to prevent cross-origin requests, while also removing any external dependencies, although it tends to increase files sizes.
-
-
-
-๐ผ๏ธ Templates
+๐ฌ Creating SVGs images on-the-fly
-Templates requires you to be comfortable with HTML, CSS and JavaScript ([EJS](https://github.com/mde/ejs) flavored). +Metrics actually exploit the possibility of integrating HTML and CSS into SVGs, so basically creating these images is as simple as designing static web pages. It can even handle animations and transparency. -Metrics does not really accept contributions on [default templates](https://github.com/lowlighter/metrics/tree/master/source/templates) in order to avoid bloating main repository with a lot of templates, but fear not! Users will still be able to use your custom templates thanks to [community templates](source/templates/community)! + -If you make something awesome, don't hesistate to share it! +SVGs are templated through [EJS framework](https://github.com/mde/ejs) to make the whole rendering process easier thanks to variables, conditional and loop statements. Only drawback is that it tends to make syntax coloration a bit confused because templates are often misinterpreted as HTML tags markers (`<%= "EJS templating syntax" %>`). -For a quick start, use: -```shell -npm run quickstart -- template
-
๐ฌ Creating a new template from scratch
+Since SVG renders differently depending on OS and browsers (system fonts, CSS support, ...), it's pretty hard to compute dynamically height. Previously, it was computed with ugly formulas, but as it wasn't scaling really well (especially since the introduction of variable content length plugins). It was often resulting in large empty blank spaces or really badly cropped image. -Find a cool name for your template and create an eponym folder in [`source/templates`](https://github.com/lowlighter/metrics/tree/master/source/templates). +To solve this, metrics now spawns a [puppeteer](https://github.com/puppeteer/puppeteer) instance and directly render SVG in a browser environment (with all animations disabled). An hidden "marker" element is placed at the end of the image, and is used to resize image through its Y-offset. -Then, you'll need to create the following files: -- `README.md` will contain template description and documentation -- `image.svg` will contain the base render structure of your template -- `partials/` is a folder that'll contain parts of your template (called "partials") - - `partials/_.json` is a JSON array which lists your partials (these will be displayed in the same order as listed, unless if overriden by user with `config_order` option) + -The following files are optional: -- `fonts.css` can contain your custom fonts (base64 encoded) if needed -- `styles.css` can contain your CSS that'll style your template -- `template.mjs` can contain additional data processing and formatting at template-level +Additional bonus of using pupeeter is that it can take screenshots, making it easy to convert SVGs to PNG output. -Optional files will fallback to the one defined in [`classic`](https://github.com/lowlighter/metrics/tree/master/source/templates/classic) template if unexistant. - -Note that by default, `template.mjs` is skipped when using official release with community templates, to prevent malicious code to leaks token and credentials. +Finally, SVGs image can be optimized through [svgo](https://github.com/svg/svgo), which helps to remove unused attributes and blank space, while also reducing a bit the file size.
-๐ฌ Creating a
+
-
+As for other external services (Twitter, Spotify, PageSpeed, ...), metrics use their respective APIs, usually making https requests through [axios](https://github.com/axios/axios) and by following their documentation. It would be overkill to install entire SDKs for these since plugins rarely uses more than 2/3 calls.
-#### โน๏ธ Examples workflows
-
-'''yaml
-- uses: lowlighter/metrics@latest
- with:
- # ... other options
- setup_community_templates: user/metrics@master:template
- template: "@template"
-'''
-
-```
-
-
-
-
-๐ฌ Creating a README.md
+๐ฌ Gathering external data from GitHub APIs and Third-Party services
-Your `README.md` will document your template and explain how it works. -It must contain at least the following: -```markdown -### ๐ My custom template +Metrics mostly use GitHub APIs since it is its primary target. Most of the time, data are retrieved through GraphQL to save APIs requests, but it sometimes fallback on REST for other features. Octokit SDKs are used to make it easier. -|
- |
-
-๐ฌ Creating
-
-Once you finished setting up template folder structure, paste the following in `image.svg` to get started:
-```html
-
-```
-
-Let's explain what it does.
-
-`fonts` and `style` variables will be populated with the same content as your `fonts.css` and `styles.css` files.
-Like said previously, if these does not exists, it'll contain the same content as the `classic` template files.
-
-The main loop will iterate on `partials` variable which contains your partials index set in `_.json`.
-
-Finally, you may have noticed that `height` is set to a very high number, and that there is a `#metrics-end` element at the bottom of the SVG template. This is because rendered height is computed dynamically through a [puppeteer](https://github.com/puppeteer/puppeteer) browser instance which locate `#metrics-end` and use its *y-coordinate* and `config_padding` to set final height. So you should leave it like this to ensure your rendered image will be correctly sized.
+In last resort, pupeeter is seldom used to scrap websites, though its use tends to make things slow and unstable (as it'll break upon HTML structural changes).
๐ฌ Creating image.svg
-
-Once you finished setting up template folder structure, paste the following in `image.svg` to get started:
-```html
-
-```
-
-Let's explain what it does.
-
-`fonts` and `style` variables will be populated with the same content as your `fonts.css` and `styles.css` files.
-Like said previously, if these does not exists, it'll contain the same content as the `classic` template files.
-
-The main loop will iterate on `partials` variable which contains your partials index set in `_.json`.
-
-Finally, you may have noticed that `height` is set to a very high number, and that there is a `#metrics-end` element at the bottom of the SVG template. This is because rendered height is computed dynamically through a [puppeteer](https://github.com/puppeteer/puppeteer) browser instance which locate `#metrics-end` and use its *y-coordinate* and `config_padding` to set final height. So you should leave it like this to ensure your rendered image will be correctly sized.
+In last resort, pupeeter is seldom used to scrap websites, though its use tends to make things slow and unstable (as it'll break upon HTML structural changes).
-
๐ฌ Customizing templates with partials
+๐ฌ Web instance and GitHub action similarities
-Partials are sections that'll be displayed in rendered metrics. +Historically, metrics used to be only a web service without any customization possible. The single input was a GitHub username, and was composed of what is now `base` content (along with `languages` and `followup` plugin, which is why they can be computed without any additional queries). That's why `base` content is handled a bit differently from plugins. -It's just HTML with CSS which can be templated through [EJS](https://github.com/mde/ejs) framework. -Basically, you can use JavaScript statements in templating tags (`<% %>`) to display variables content and to programmatically create content. +As it gathered more and more plugins over time, generating a single user's metrics was becoming costly both in terms of resources but also in APIs requests. It was thus decided to switch to GitHub Action. At first, it was just a way to explore possibilities of this GitHub feature, but now it's basically the full-experience of metrics (unless you use your own self-hosted instance). + +Both web instance and Action actually use the same entrypoint so they basically have the same features. +Action just format inputs into a query-like object (similarly to when url params are parsed by web instance), from which metrics compute the rendered image. It also makes testing easier, as test cases can be reused since only inputs differs.
-
-
-๐ฌ Using custom fonts
+๐ฌ Testing and mocking
-This is actually not recommended because it drastically increases the size of generated metrics, but it should also make your rendering more consistant. The trick is to actually restrict the charset used to keep file size small. +Testing is done through [jest](https://github.com/facebook/jest) framework. -Below is a simplified process on how to generate base64 encoded fonts to use in metrics: -- 1. Find a font on [fonts.google.com](https://fonts.google.com/) - - Select regular, bold, italic and bold+italic fonts - - Open `embed` tab and extract the `href` -- 2. Open extracted `href` and append `&text=` params with used characters from SVG - - e.g. `&text=%26%27"%7C%60%5E%40ยฐ%3F!%23%24%25()*%2B%2C-.%2F0123456789%3A%3B<%3D>ABCDEFGHIJKLMNOPQRSTUVWXYZ%5B%5D_abcdefghijklmnopqrstuvwxyz%7B%7D~โโโโโกโ` -- 3. Download each font file from url links from the generated stylesheet -- 4. Convert them into base64 with `woff` extension on [transfonter.org]https://transfonter.org/) and download archive -- 5. Extract archive and copy the content of the generated stylesheet to `fonts.css` -- 6. Update your template - - Include `
-
-```
-
-
๐งฉ Plugins
- -Plugins are self-sufficient and independant code functions that gather additional data from GitHub APIs or external sources. - -
-
-
-For a quick start, use:
-```shell
-npm run quickstart -- plugin ๐ฌ Plugin guidelines
- -- A plugin should never be dependent on others plugins - - But they're allowed to use data gathered by main metrics function -- Avoid the need of new external dependencies (like SDKs) - - Most of the time, SDKs are overkill when a few HTTP calls do the trick - - `imports` probably contains a library that can help you achieving what you want -- Avoid using raw command when possible (like spawning sub-process) - - Sub-process should be platform agnostic (i.e. working on most OS) -- Errors should always be handled gracefully by displaying an error message when it fails - - When possible, try to display explicit error messages - -
-
-
-๐ฌ Creating a new plugin
- -Find a cool word to name your plugin and create an eponym folder in [`source/plugins`](https://github.com/lowlighter/metrics/tree/master/source/plugins) folder. - -You'll also need to find an unused [emoji](https://emojipedia.org) that you'll be able to use as your plugin icon. - -Then create an `index.mjs` in your plugin folder and paste the following code: -```js -//Setup - export default async function ({login, q, imports, data, computed, rest, graphql, queries, account}, {enabled = false} = {}) { - //Plugin execution - try { - //Check if plugin is enabled and requirements are met - if ((!enabled)||(!q/* your plugin name */)) - return null - //Results - return {} - } - //Handle errors - catch (error) { - throw {error:{message:"An error occured", instance:error}} - } - } -``` - -The following inputs are available: -- `login` is set to GitHub login -- `q` contains all query parameters -- `imports` contains libraries and utilitaries that are shared amongst plugins - - `imports.url` refers to [NodeJS `url` library](https://nodejs.org/api/url.html) - - `imports.os` refers to [NodeJS `os` library](https://nodejs.org/api/os.html) - - `imports.fs` refers to [NodeJS `fs` library](https://nodejs.org/api/fs.html) - - `imports.paths` refers to [NodeJS `paths` library](https://nodejs.org/api/paths.html) - - `imports.util` refers to [NodeJS `util` library](https://nodejs.org/api/util.html) - - `imports.imgb64` refers to [renanbastos93/image-to-base64](https://github.com/renanbastos93/image-to-base64) - - `imports.axios` refers to [axios/axios](https://github.com/axios/axios) - - `imports.puppeteer` refers to [puppeteer/puppeteer](https://github.com/puppeteer/puppeteer) - - `imports.run` is an helper to run raw command - - `imports.shuffle` is an helper to shuffle array - - `imports.__module` is an helper to find `__dirname` from a module url - - And more... -- `data` and `computed` contains all data (and computed data) gathered by various APIs from main metrics function -- `graphql` and `rest` contains [octokit clients](https://github.com/octokit) for GitHub API -- `queries` contains autoloaded GraphQL queries with replacers -- `account` contains the type of account being worked on ("user" or "organization") - -The second input contains configuration settings from [settings.json](https://github.com/lowlighter/metrics/blob/master/settings.example.json), which is mostly used by web instances. - -Content of these parameters should **never** be edited directly, as your plugin should only return a new result. - -Plugins are autoloaded so you do not need to do anything special to register them. - -
-
-
-๐ฌ Gathering new data from GitHub APIs and from Third-Party services
- -For GitHub related data, always try to use their [GraphQL API](https://docs.github.com/en/graphql) or their [REST API](https://docs.github.com/en/rest) when possible. Use `puppeteer` in last resort. - -When using GraphQL API, `queries` object autoloads queries from your plugin `queries` directory and will replace all strings prefixed by a dollar sign (`$`) with eponym variables. - -For example: -```js -//Calling this - await graphql(queries.myquery({login:"github-user", account:"user"})) - -//With this in source/queries/myquery.graphql - query MyQuery { - $account(login: "$login") { - name - } - } - -//Will have the same result as calling this - await graphql(` - query MyQuery { - user(login: "github-user") { - name - } - } - `) -``` - -For REST API, check out their [documentation](https://octokit.github.io/rest.js/v18/). - -As for Third-Party services, always prefer using their APIs (you can use [`imports.axios`](https://github.com/axios/axios) for easy HTTP requests) when they exists before having recourse to [`imports.puppeteer`](https://github.com/puppeteer/puppeteer). - -New external dependencies should be avoided at all costs, especially since most of the time it's overkill to setup a new SDK. - -
-
-
-<% } %>
-```
-
-Let's explain what it does.
-
-First conditional statement will ensure that your partial only execute when your plugin is enabled.
-
-The nested one will check if your plugin resulted in an error, and if that's the case, it'll display an error message instead.
-Else, if it's successful, you'll get the second section in render.
-
-Plugins errors should always be handled gracefully when possible.
-
-If you need additional CSS rules, edits the `style.css` of edited template.
-
-
-
-๐ฌ Creating a partial to display your plugin result
- -Create new files in `partials` of `source/templates` you want to support with `.ejs` extension. - -You can paste the following for a quick start: -```html -<% if (plugins./* your plugin name */) { %> -
- <% if (plugins./* your plugin name */.error) { %>
-
-
- <% } else { %>
-
- <%# Do stuff in there -%>
-
- <% } %>
-
-
-
- <%= plugins./* your plugin name */.error.message %>
-
-
-
-
-๐ฌ Fast prototyping with web instance
- -The easiest way to test and prototype your plugin is to use a web instance. - -Configure a [settings.json](https://github.com/lowlighter/metrics/blob/master/settings.example.json) with a valid GitHub token and with debug mode enabled. -Then start a web instance with `npm start` (you may have to run `npm install` if that's the first time you use the web instance). - -Then try to generate your metrics in your browser with your GitHub user and your plugin enabled, and see if it works as expected: -``` -http://localhost:3000/your-github-login?base=0&your-plugin-name=1 -``` - -
-๐ฌ Registering plugin options in
-
-`metadata.yml` is a special file that will be used to parse user inputs and to generate final `action.yml`
-
-```yaml
-name: "๐งฉ Your plugin name"
-
-# Estimate of how many GitHub requests will be used
-cost: N/A
-
-# Supported modes
-supports:
- - user
- - organization
- - repository
-
-# Inputs list
-inputs:
-
- # Enable or disable plugin
- plugin_custom:
- description: Your custom plugin
- type: boolean
- default: no
-```
-
-The following types are supported:
-```yaml
-string:
- type: string
-
-select:
- type: string
- values:
- - allowed-value-1
- - allowed-value-2
- - ...
-
-boolean:
- type: boolean
-
-number:
- type: number
-
-ranged:
- type: number
- min: 0
- max: 100
-
-array:
- type: array
- format: comma-separated
-
-array_select:
- type: array
- format: comma-separated
- values:
- - allowed-value-1
- - allowed-value-2
- - ...
-
-json:
- type: json
-```
-
-
-
-๐ฌ Registering plugin options in metadata.yml
-
-`metadata.yml` is a special file that will be used to parse user inputs and to generate final `action.yml`
-
-```yaml
-name: "๐งฉ Your plugin name"
-
-# Estimate of how many GitHub requests will be used
-cost: N/A
-
-# Supported modes
-supports:
- - user
- - organization
- - repository
-
-# Inputs list
-inputs:
-
- # Enable or disable plugin
- plugin_custom:
- description: Your custom plugin
- type: boolean
- default: no
-```
-
-The following types are supported:
-```yaml
-string:
- type: string
-
-select:
- type: string
- values:
- - allowed-value-1
- - allowed-value-2
- - ...
-
-boolean:
- type: boolean
-
-number:
- type: number
-
-ranged:
- type: number
- min: 0
- max: 100
-
-array:
- type: array
- format: comma-separated
-
-array_select:
- type: array
- format: comma-separated
- values:
- - allowed-value-1
- - allowed-value-2
- - ...
-
-json:
- type: json
-```
-
-
-
-
-๐ฌ Create mocked data and tests
- -Creating tests for your plugin ensure that external changes don't break it. - -You can define your tests cases in `tests.yml` in your plugin directory, which will automatically test your plugin with: - - Metrics action - - Metrics web instance - - Metrics web instance placeholder (rendered by browser) - -As most of APIs (including GitHub) usually have a rate-limit to ensure quality of their service. -To bypass these restrictions but still perform tests, you must mock their data which simulates APIs call returns. - -Add them in [`source/app/mocks/api/`](/source/app/mocks/api) folder. - -If you're using `axios` or GitHub GraphQL API, these files are autoloaded so you just need to create new functions (see other mocked data for examples). - -If you're using GitHub REST API, add your mocks in [`source/app/mocks/rest`](/source/app/mocks/rest) with same path as octokit. - -
-๐ฌ Creating a
-
-Your `README.md` will document your plugin and explain how it works.
-It must contain at least the following:
-
-```markdown
-### ๐งฉ Your plugin name
-
-
-
-
-#### โน๏ธ Examples workflows
-
-[โก๏ธ Available options for this plugin](metadata.yml)
-
-'''yaml
-- uses: lowlighter/metrics@latest
- with:
- # ... other options
- plugin_custom: yes
-'''
-
-```
-
-Note that you **must** keep `` tags as these will be extracted to autogenerated global `README.md` with your example.
-
-
+While the best would be to work with real data during testing, to avoid consuming too much APIs requests for testing (and to be more planet friendly), they're [mocked](https://github.com/lowlighter/metrics/blob/master/source/app/mocks.mjs) using [JavaScript Proxies](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy) and [Faker.js](https://github.com/marak/Faker.js/). Basically function calls are "trapped" and send randomly generated data from Faker.js if we're in a development environment.
diff --git a/source/app/metrics/utils.mjs b/source/app/metrics/utils.mjs
index ae680a08..c0dc2bab 100644
--- a/source/app/metrics/utils.mjs
+++ b/source/app/metrics/utils.mjs
@@ -117,7 +117,9 @@
console.debug(`metrics/svgresize > padding width*${padding.width}, height*${padding.height}`)
//Render through browser and resize height
const page = await svgresize.browser.newPage()
- await page.setContent(svg, {waitUntil:"load"})
+ await page.setContent(svg, {waitUntil:["load", "domcontentloaded", "networkidle2"]})
+ await page.addStyleTag({content:"body { margin: 0; padding: 0; }"})
+ await wait(1)
let mime = "image/svg+xml"
let {resized, width, height} = await page.evaluate(async padding => {
//Disable animations
diff --git a/source/plugins/activity/metadata.yml b/source/plugins/activity/metadata.yml
index 7f2fc465..69b9bb25 100644
--- a/source/plugins/activity/metadata.yml
+++ b/source/plugins/activity/metadata.yml
@@ -1,5 +1,6 @@
name: "๐ฐ Recent activity"
cost: 1 REST request per 100 events
+categorie: github
supports:
- user
- organization
diff --git a/source/plugins/anilist/metadata.yml b/source/plugins/anilist/metadata.yml
index 37efcfbf..195592aa 100644
--- a/source/plugins/anilist/metadata.yml
+++ b/source/plugins/anilist/metadata.yml
@@ -1,5 +1,6 @@
name: "๐ธ Anilist"
cost: N/A
+categorie: social
supports:
- user
- organization
diff --git a/source/plugins/base/README.md b/source/plugins/base/README.md
index a2c0c2b7..ab8acaac 100644
--- a/source/plugins/base/README.md
+++ b/source/plugins/base/README.md
@@ -32,7 +32,8 @@ These are all enabled by default, but you can explicitely opt out from them.
- uses: lowlighter/metrics@latest
with:
# ... other options
- base: header, repositories # Only display "header" and "repositories" sections
- repositories: 100 # Query only last 100 repositories
- repositories_forks: no # Don't include forks
+ base: header, repositories # Only display "header" and "repositories" sections
+ repositories: 100 # Query only last 100 repositories
+ repositories_forks: no # Don't include forks
+ repositories_affiliations: owner # Display only repositories where user is owner
```
diff --git a/source/plugins/base/index.mjs b/source/plugins/base/index.mjs
index 00427100..07b1426f 100644
--- a/source/plugins/base/index.mjs
+++ b/source/plugins/base/index.mjs
@@ -7,7 +7,7 @@
export default async function({login, graphql, data, q, queries, imports}, conf) {
//Load inputs
console.debug(`metrics/compute/${login}/base > started`)
- let {repositories, repositories_forks:forks} = imports.metadata.plugins.base.inputs({data, q, account:"bypass"}, {repositories:conf.settings.repositories ?? 100})
+ let {repositories, "repositories.forks":forks, "repositories.affiliations":affiliations} = imports.metadata.plugins.base.inputs({data, q, account:"bypass"}, {repositories:conf.settings.repositories ?? 100})
//Skip initial data gathering if not needed
if (conf.settings.notoken)
@@ -23,7 +23,7 @@
try {
//Query data from GitHub API
console.debug(`metrics/compute/${login}/base > account ${account}`)
- const queried = await graphql(queries.base[account]({login, "calendar.from":new Date(Date.now()-14*24*60*60*1000).toISOString(), "calendar.to":(new Date()).toISOString(), forks:forks ? "" : ", isFork: false"}))
+ const queried = await graphql(queries.base[account]({login, "calendar.from":new Date(Date.now()-14*24*60*60*1000).toISOString(), "calendar.to":(new Date()).toISOString(), forks:forks ? "" : ", isFork: false", affiliations:affiliations ? `, ownerAffiliations: ${affiliations.toLocaleUpperCase()}` : ""}))
Object.assign(data, {user:queried[account]})
postprocess?.[account]({login, data})
//Query repositories from GitHub API
@@ -33,7 +33,7 @@
let pushed = 0
do {
console.debug(`metrics/compute/${login}/base > retrieving repositories after ${cursor}`)
- const {[account]:{repositories:{edges, nodes}}} = await graphql(queries.base.repositories({login, account, after:cursor ? `after: "${cursor}"` : "", repositories:Math.min(repositories, {user:100, organization:25}[account]), forks:forks ? "" : ", isFork: false"}))
+ const {[account]:{repositories:{edges, nodes}}} = await graphql(queries.base.repositories({login, account, after:cursor ? `after: "${cursor}"` : "", repositories:Math.min(repositories, {user:100, organization:25}[account]), forks:forks ? "" : ", isFork: false", affiliations:affiliations ? `, ownerAffiliations: ${affiliations.toLocaleUpperCase()}` : ""}))
cursor = edges?.[edges?.length-1]?.cursor
data.user.repositories.nodes.push(...nodes)
pushed = nodes.length
diff --git a/source/plugins/base/metadata.yml b/source/plugins/base/metadata.yml
index 97bfd60b..c6f32ed1 100644
--- a/source/plugins/base/metadata.yml
+++ b/source/plugins/base/metadata.yml
@@ -1,5 +1,6 @@
name: "๐๏ธ Base content"
cost: 1 GraphQL request
+categorie: core
supports:
- user
- organization
@@ -32,3 +33,13 @@ inputs:
description: Include forks in metrics
type: boolean
default: no
+
+ # Filter repositories by user affiliations
+ repositories_affiliations:
+ description: Repositories affiliations
+ type: string
+ default: ""
+ values:
+ - owner
+ - collaborator
+ - organization_member
diff --git a/source/plugins/base/queries/repositories.graphql b/source/plugins/base/queries/repositories.graphql
index b2b4984c..cf1ec688 100644
--- a/source/plugins/base/queries/repositories.graphql
+++ b/source/plugins/base/queries/repositories.graphql
@@ -1,6 +1,6 @@
query BaseRepositories {
$account(login: "$login") {
- repositories($after first: $repositories $forks, orderBy: {field: UPDATED_AT, direction: DESC}) {
+ repositories($after first: $repositories $forks $affiliations, orderBy: {field: UPDATED_AT, direction: DESC}) {
edges {
cursor
}
diff --git a/source/plugins/base/queries/user.graphql b/source/plugins/base/queries/user.graphql
index 19eca20a..667ee62d 100644
--- a/source/plugins/base/queries/user.graphql
+++ b/source/plugins/base/queries/user.graphql
@@ -8,7 +8,7 @@ query BaseUser {
websiteUrl
isHireable
twitterUsername
- repositories(last: 0 $forks) {
+ repositories(last: 0 $forks $affiliations) {
totalCount
totalDiskUsage
nodes {
diff --git a/source/plugins/core/metadata.yml b/source/plugins/core/metadata.yml
index 415da242..a6eb243f 100644
--- a/source/plugins/core/metadata.yml
+++ b/source/plugins/core/metadata.yml
@@ -1,5 +1,6 @@
name: "๐งฑ Core"
cost: N/A
+categorie: core
supports:
- user
- organization
diff --git a/source/plugins/followup/metadata.yml b/source/plugins/followup/metadata.yml
index 73e3e28f..1dd6f6f5 100644
--- a/source/plugins/followup/metadata.yml
+++ b/source/plugins/followup/metadata.yml
@@ -1,5 +1,6 @@
name: "๐๏ธ Follow-up of issues and pull requests"
cost: 0 API request
+categorie: github
supports:
- user
- organization
diff --git a/source/plugins/gists/metadata.yml b/source/plugins/gists/metadata.yml
index b7d7ee64..09d22aec 100644
--- a/source/plugins/gists/metadata.yml
+++ b/source/plugins/gists/metadata.yml
@@ -1,5 +1,6 @@
name: "๐ซ Gists"
cost: 1 GraphQL request per 100 gists
+categorie: github
supports:
- user
inputs:
diff --git a/source/plugins/habits/index.mjs b/source/plugins/habits/index.mjs
index 2af642e1..e91280af 100644
--- a/source/plugins/habits/index.mjs
+++ b/source/plugins/habits/index.mjs
@@ -100,7 +100,7 @@
console.debug(`metrics/compute/${login}/plugins > habits > running linguist`)
;(await imports.run(`${prefix} github-linguist --breakdown`, {cwd:path}))
//Parse linguist result
- .split("\n").map(line => line.match(/(?
๐ฌ Creating a README.md
-
-Your `README.md` will document your plugin and explain how it works.
-It must contain at least the following:
-
-```markdown
-### ๐งฉ Your plugin name
-
-|
- |
-