Merge branch 'master' into self-hosted

README.md

@@ -10,6 +10,8 @@ The underlying CodeQL CLI, used in this action, is licensed under the [GitHub Co
 
 ## Usage
 
+This is a short walkthrough, but for more information read [configuring code scanning](https://help.github.com/en/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning).
+
 To get code scanning results from CodeQL analysis on your repo you can use the following workflow as a template:
 
 ```yaml
@@ -90,24 +92,9 @@ If you prefer to integrate this within an existing CI workflow, it should end up
   uses: github/codeql-action/analyze@v1
 ```
 
-### Actions triggers
+### Configuration file
 
-The CodeQL action should be run on `push` events, and on a `schedule`. `Push` events allow us to do a detailed analysis of the delta in a pull request, while the `schedule` event ensures that GitHub regularly scans the repository for the latest vulnerabilities, even if the repository becomes inactive. This action does not support the `pull_request` event.
-
-### Configuration
-
-You may optionally specify additional queries for CodeQL to execute by using a config file. The queries must belong to a [QL pack](https://help.semmle.com/codeql/codeql-cli/reference/qlpack-overview.html) and can be in your repository or any public repository. You can choose a single .ql file, a folder containing multiple .ql files, a .qls [query suite](https://help.semmle.com/codeql/codeql-cli/procedures/query-suites.html) file, or any combination of the above. To use queries stored in your repository or from other repositories use the same syntax as when [using an action](https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstepsuses). Note that when using local queries starting with `./`, the path is relative to the root of the repository and not to the location of the config file.
-
-You can disable the default queries using `disable-default-queries: true`.
-
-You can choose to ignore some files or folders from the analysis, or include additional files/folders for analysis. This *only* works for Javascript and Python analysis.
-Identifying potential files for extraction:
-
-- Scans each folder that's defined as `paths` in turn, traversing subfolders, and looking for relevant files.
-- If it finds a subfolder that's defined as `paths-ignore`, stop traversing.
-- If a file or folder is both in `paths` and `paths-ignore`, the `paths-ignore` is ignored.
-
-Use the `config-file` parameter of the init action to enable the configuration file. For example:
+Use the `config-file` parameter of the `init` action to enable the configuration file. The value of `config-file` is the path to the configuration file you want to use. This example loads the configuration file `./.github/codeql/codeql-config.yml`.
 
 ```yaml
 - uses: github/codeql-action/init@v1
@@ -115,72 +102,8 @@ Use the `config-file` parameter of the init action to enable the configuration f
     config-file: ./.github/codeql/codeql-config.yml
 ```
 
-A config file looks like this:
-
-```yaml
-name: "My CodeQL config"
-
-disable-default-queries: true
-
-queries:
-  - name: In-repo queries (Runs the queries located in the my-queries folder of the repo)
-    uses: ./my-queries
-  - name: External Javascript QL pack (Runs a QL pack located in an external repo)
-    uses: /Semmle/ql/javascript/ql/src/Electron@master
-  - name: External query (Runs a single query located in an external QL pack)
-    uses: Semmle/ql/javascript/ql/src/AngularJS/DeadAngularJSEventListener.ql@master
-  - name: Select query suite (Runs a query suites)
-    uses: ./codeql-querypacks/complex-python-querypack/rootAndBar.qls
-
-paths:
-  - src/util.ts
-
-paths-ignore:
-  - src
-  - lib
-```
+The configuration file must be located within the local repository. For information on how to write a configuration file, see "[Using a custom configuration](https://help.github.com/en/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#using-a-custom-configuration)."
 
 ## Troubleshooting
 
-### Trouble with Go dependencies
-
-#### If you use a vendor directory
-
-Try passing
-
-```yaml
-env:
-  GOFLAGS: "-mod=vendor"
-```
-
-to `github/codeql-action/analyze`.
-
-#### If you do not use a vendor directory
-
-Dependencies on public repositories should just work. If you have dependencies on private repositories, one option is to use `git config` and a [personal access token](https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line) to authenticate when downloading dependencies. Add a section like
-
-```yaml
-steps:
-  - name: Configure git private repo access
-    env:
-      TOKEN: ${{ secrets.GITHUB_PAT }}
-    run: |
-      git config --global url."https://${TOKEN}@github.com/foo/bar".insteadOf "https://github.com/foo/bar"
-      git config --global url."https://${TOKEN}@github.com/foo/baz".insteadOf "https://github.com/foo/baz"
-```
-
-before any codeql actions. A similar thing can also be done with an SSH key or deploy key.
-
-### C# using dotnet version 2 on linux
-
-This currently requires invoking `dotnet` with the `/p:UseSharedCompilation=false` flag. For example:
-
-```shell
-dotnet build /p:UseSharedCompilation=false
-```
-
-Version 3 does not require the additional flag.
-
-### Analysing Go together with other languages on `macos-latest`
-
-When running on macos it is currently not possible to analyze Go in conjunction with any of Java, C/C++, or C#. Each language can still be analyzed separately.
+Read about [troubleshooting code scanning](https://help.github.com/en/github/finding-security-vulnerabilities-and-errors-in-your-code/troubleshooting-code-scanning).

lib/finalize-db.js

@@ -174,7 +174,7 @@ async function run() {
         await runQueries(codeqlCmd, databaseFolder, sarifFolder, config);
         if ('true' === core.getInput('upload')) {
             if (!await upload_lib.upload(sarifFolder)) {
-                await util.reportActionFailed('failed', 'upload');
+                await util.reportActionFailed('finish', 'upload');
                 return;
             }
         }

lib/setup-tracer.js

@@ -206,8 +206,8 @@ async function run() {
         await util.reportActionFailed('init', error.message, error.stack);
         return;
     }
-    core.exportVariable(sharedEnv.CODEQL_ACTION_INIT_COMPLETED, 'true');
     await util.reportActionSucceeded('init');
+    core.exportVariable(sharedEnv.CODEQL_ACTION_INIT_COMPLETED, 'true');
 }
 run().catch(e => {
     core.setFailed("init action failed: " + e);

lib/upload-lib.js

@@ -125,58 +125,52 @@ exports.countResultsInSarif = countResultsInSarif;
 // Returns true iff the upload occurred and succeeded
 async function uploadFiles(sarifFiles) {
     core.startGroup("Uploading results");
-    let succeeded = false;
-    try {
-        const sentinelEnvVar = "CODEQL_UPLOAD_SARIF";
-        if (process.env[sentinelEnvVar]) {
-            core.error("Aborting upload: only one run of the codeql/analyze or codeql/upload-sarif actions is allowed per job");
-            return false;
-        }
-        core.exportVariable(sentinelEnvVar, sentinelEnvVar);
-        const commitOid = await util.getCommitOid();
-        const workflowRunIDStr = util.getRequiredEnvParam('GITHUB_RUN_ID');
-        const ref = util.getRef();
-        const analysisKey = await util.getAnalysisKey();
-        const analysisName = util.getRequiredEnvParam('GITHUB_WORKFLOW');
-        const startedAt = process.env[sharedEnv.CODEQL_ACTION_STARTED_AT];
-        core.info("Uploading sarif files: " + JSON.stringify(sarifFiles));
-        let sarifPayload = combineSarifFiles(sarifFiles);
-        sarifPayload = fingerprints.addFingerprints(sarifPayload);
-        const zipped_sarif = zlib_1.default.gzipSync(sarifPayload).toString('base64');
-        let checkoutPath = core.getInput('checkout_path');
-        let checkoutURI = file_url_1.default(checkoutPath);
-        const workflowRunID = parseInt(workflowRunIDStr, 10);
-        if (Number.isNaN(workflowRunID)) {
-            core.setFailed('GITHUB_RUN_ID must define a non NaN workflow run ID');
-            return false;
-        }
-        let matrix = core.getInput('matrix');
-        if (matrix === "null" || matrix === "") {
-            matrix = undefined;
-        }
-        const toolNames = util.getToolNames(sarifPayload);
-        const payload = JSON.stringify({
-            "commit_oid": commitOid,
-            "ref": ref,
-            "analysis_key": analysisKey,
-            "analysis_name": analysisName,
-            "sarif": zipped_sarif,
-            "workflow_run_id": workflowRunID,
-            "checkout_uri": checkoutURI,
-            "environment": matrix,
-            "started_at": startedAt,
-            "tool_names": toolNames,
-        });
-        // Log some useful debug info about the info
-        core.debug("Raw upload size: " + sarifPayload.length + " bytes");
-        core.debug("Base64 zipped upload size: " + zipped_sarif.length + " bytes");
-        core.debug("Number of results in upload: " + countResultsInSarif(sarifPayload));
-        // Make the upload
-        succeeded = await uploadPayload(payload);
+    const sentinelEnvVar = "CODEQL_UPLOAD_SARIF";
+    if (process.env[sentinelEnvVar]) {
+        core.error("Aborting upload: only one run of the codeql/analyze or codeql/upload-sarif actions is allowed per job");
+        return false;
     }
-    catch (error) {
-        core.setFailed(error.message);
+    core.exportVariable(sentinelEnvVar, sentinelEnvVar);
+    const commitOid = await util.getCommitOid();
+    const workflowRunIDStr = util.getRequiredEnvParam('GITHUB_RUN_ID');
+    const ref = util.getRef();
+    const analysisKey = await util.getAnalysisKey();
+    const analysisName = util.getRequiredEnvParam('GITHUB_WORKFLOW');
+    const startedAt = process.env[sharedEnv.CODEQL_ACTION_STARTED_AT];
+    core.info("Uploading sarif files: " + JSON.stringify(sarifFiles));
+    let sarifPayload = combineSarifFiles(sarifFiles);
+    sarifPayload = fingerprints.addFingerprints(sarifPayload);
+    const zipped_sarif = zlib_1.default.gzipSync(sarifPayload).toString('base64');
+    let checkoutPath = core.getInput('checkout_path');
+    let checkoutURI = file_url_1.default(checkoutPath);
+    const workflowRunID = parseInt(workflowRunIDStr, 10);
+    if (Number.isNaN(workflowRunID)) {
+        core.setFailed('GITHUB_RUN_ID must define a non NaN workflow run ID');
+        return false;
     }
+    let matrix = core.getInput('matrix');
+    if (matrix === "null" || matrix === "") {
+        matrix = undefined;
+    }
+    const toolNames = util.getToolNames(sarifPayload);
+    const payload = JSON.stringify({
+        "commit_oid": commitOid,
+        "ref": ref,
+        "analysis_key": analysisKey,
+        "analysis_name": analysisName,
+        "sarif": zipped_sarif,
+        "workflow_run_id": workflowRunID,
+        "checkout_uri": checkoutURI,
+        "environment": matrix,
+        "started_at": startedAt,
+        "tool_names": toolNames,
+    });
+    // Log some useful debug info about the info
+    core.debug("Raw upload size: " + sarifPayload.length + " bytes");
+    core.debug("Base64 zipped upload size: " + zipped_sarif.length + " bytes");
+    core.debug("Number of results in upload: " + countResultsInSarif(sarifPayload));
+    // Make the upload
+    const succeeded = await uploadPayload(payload);
     core.endGroup();
     return succeeded;
 }

src/finalize-db.ts

@@ -218,7 +218,7 @@ async function run() {
 
     if ('true' === core.getInput('upload')) {
       if (!await upload_lib.upload(sarifFolder)) {
-        await util.reportActionFailed('failed', 'upload');
+        await util.reportActionFailed('finish', 'upload');
         return;
       }
     }

src/setup-tracer.ts

@@ -239,8 +239,8 @@ async function run() {
         await util.reportActionFailed('init', error.message, error.stack);
         return;
     }
-    core.exportVariable(sharedEnv.CODEQL_ACTION_INIT_COMPLETED, 'true');
     await util.reportActionSucceeded('init');
+    core.exportVariable(sharedEnv.CODEQL_ACTION_INIT_COMPLETED, 'true');
 }
 
 run().catch(e => {

src/upload-lib.ts

@@ -127,67 +127,63 @@ export function countResultsInSarif(sarif: string): number {
 // Returns true iff the upload occurred and succeeded
 async function uploadFiles(sarifFiles: string[]): Promise<boolean> {
     core.startGroup("Uploading results");
-    let succeeded = false;
-    try {
-        const sentinelEnvVar = "CODEQL_UPLOAD_SARIF";
-        if (process.env[sentinelEnvVar]) {
-            core.error("Aborting upload: only one run of the codeql/analyze or codeql/upload-sarif actions is allowed per job");
-            return false;
-        }
-        core.exportVariable(sentinelEnvVar, sentinelEnvVar);
 
-        const commitOid = await util.getCommitOid();
-        const workflowRunIDStr = util.getRequiredEnvParam('GITHUB_RUN_ID');
-        const ref = util.getRef();
-        const analysisKey = await util.getAnalysisKey();
-        const analysisName = util.getRequiredEnvParam('GITHUB_WORKFLOW');
-        const startedAt = process.env[sharedEnv.CODEQL_ACTION_STARTED_AT];
-
-        core.info("Uploading sarif files: " + JSON.stringify(sarifFiles));
-        let sarifPayload = combineSarifFiles(sarifFiles);
-        sarifPayload = fingerprints.addFingerprints(sarifPayload);
-
-        const zipped_sarif = zlib.gzipSync(sarifPayload).toString('base64');
-        let checkoutPath = core.getInput('checkout_path');
-        let checkoutURI = fileUrl(checkoutPath);
-        const workflowRunID = parseInt(workflowRunIDStr, 10);
-
-        if (Number.isNaN(workflowRunID)) {
-            core.setFailed('GITHUB_RUN_ID must define a non NaN workflow run ID');
-            return false;
-        }
-
-        let matrix: string | undefined = core.getInput('matrix');
-        if (matrix === "null" || matrix === "") {
-            matrix = undefined;
-        }
-
-        const toolNames = util.getToolNames(sarifPayload);
-
-        const payload = JSON.stringify({
-            "commit_oid": commitOid,
-            "ref": ref,
-            "analysis_key": analysisKey,
-            "analysis_name": analysisName,
-            "sarif": zipped_sarif,
-            "workflow_run_id": workflowRunID,
-            "checkout_uri": checkoutURI,
-            "environment": matrix,
-            "started_at": startedAt,
-            "tool_names": toolNames,
-        });
-
-        // Log some useful debug info about the info
-        core.debug("Raw upload size: " + sarifPayload.length + " bytes");
-        core.debug("Base64 zipped upload size: " + zipped_sarif.length + " bytes");
-        core.debug("Number of results in upload: " + countResultsInSarif(sarifPayload));
-
-        // Make the upload
-        succeeded = await uploadPayload(payload);
-
-    } catch (error) {
-        core.setFailed(error.message);
+    const sentinelEnvVar = "CODEQL_UPLOAD_SARIF";
+    if (process.env[sentinelEnvVar]) {
+        core.error("Aborting upload: only one run of the codeql/analyze or codeql/upload-sarif actions is allowed per job");
+        return false;
     }
+    core.exportVariable(sentinelEnvVar, sentinelEnvVar);
+
+    const commitOid = await util.getCommitOid();
+    const workflowRunIDStr = util.getRequiredEnvParam('GITHUB_RUN_ID');
+    const ref = util.getRef();
+    const analysisKey = await util.getAnalysisKey();
+    const analysisName = util.getRequiredEnvParam('GITHUB_WORKFLOW');
+    const startedAt = process.env[sharedEnv.CODEQL_ACTION_STARTED_AT];
+
+    core.info("Uploading sarif files: " + JSON.stringify(sarifFiles));
+    let sarifPayload = combineSarifFiles(sarifFiles);
+    sarifPayload = fingerprints.addFingerprints(sarifPayload);
+
+    const zipped_sarif = zlib.gzipSync(sarifPayload).toString('base64');
+    let checkoutPath = core.getInput('checkout_path');
+    let checkoutURI = fileUrl(checkoutPath);
+    const workflowRunID = parseInt(workflowRunIDStr, 10);
+
+    if (Number.isNaN(workflowRunID)) {
+        core.setFailed('GITHUB_RUN_ID must define a non NaN workflow run ID');
+        return false;
+    }
+
+    let matrix: string | undefined = core.getInput('matrix');
+    if (matrix === "null" || matrix === "") {
+        matrix = undefined;
+    }
+
+    const toolNames = util.getToolNames(sarifPayload);
+
+    const payload = JSON.stringify({
+        "commit_oid": commitOid,
+        "ref": ref,
+        "analysis_key": analysisKey,
+        "analysis_name": analysisName,
+        "sarif": zipped_sarif,
+        "workflow_run_id": workflowRunID,
+        "checkout_uri": checkoutURI,
+        "environment": matrix,
+        "started_at": startedAt,
+        "tool_names": toolNames,
+    });
+
+    // Log some useful debug info about the info
+    core.debug("Raw upload size: " + sarifPayload.length + " bytes");
+    core.debug("Base64 zipped upload size: " + zipped_sarif.length + " bytes");
+    core.debug("Number of results in upload: " + countResultsInSarif(sarifPayload));
+
+    // Make the upload
+    const succeeded = await uploadPayload(payload);
+
     core.endGroup();
 
     return succeeded;