SCORED '23

Proceedings of the 2023 Workshop on Software Supply Chain Offensive Research and Ecosystem Defenses
Last Update : [26 November, 2023]

SESSION: Keynote Talks

The Knockout of the Century: Cyber Resiliency through the Arts
  • Yesenia Yser

Join us for a captivating keynote that will take you on a unique journey where the worlds of technology, martial arts, and open-source software community empowerment converge. Discover the power of partnership within the tech industry as we unveil innovative collaborations that have reshaped the cybersecurity landscape. Learn how principles borrowed from martial arts philosophy find practical application in the ever-evolving battle against cyber threats. Be inspired by the fusion of tech, martial arts wisdom, open-source innovation, and inclusivity for a resilient cyber world. Don't miss "The Knockout of the Century" in cybersecurity!

Open Source Supply Chain Security at Google
  • Russ Cox

Google has many active efforts to improve supply chain security for open source software. This talk will present how we think about supply chain security as well as results from specific efforts, including a focus on how we land everyday impact with the millions of developers who use the Go programming language and environment.

SESSION: Technical Session: AI to the Rescue!

An Empirical Study on Using Large Language Models to Analyze Software Supply Chain Security Failures
  • Tanmay Singla
  • Dharun Anandayuvaraj
  • Kelechi G. Kalu
  • Taylor R. Schorlemmer
  • James C. Davis

As we increasingly depend on software systems, the consequences of breaches in the software supply chain become more severe. High-profile cyber attacks like SolarWinds and ShadowHammer have resulted in significant financial and data losses, underlining the need for stronger cybersecurity. One way to prevent future breaches is by studying past failures. However, traditional methods of analyzing past failures require manually reading and summarizing reports about them. Automated support could reduce costs and allow analysis of more failures. Natural Language Processing (NLP) techniques such as Large Language Models (LLMs) could be leveraged to assist the analysis of failures. In this study, we assessed the ability of Large Language Models (LLMs) to analyze historical software supply chain breaches. We used LLMs to replicate the manual analysis of 69 software supply chain security failures performed by members of the Cloud Native Computing Foundation (CNCF). We developed prompts for LLMs to categorize these by four dimensions: type of compromise, intent, nature, and impact. GPT 3.5's categorizations had an average accuracy of 68% and Bard's had an accuracy of 58% over these dimensions. We report that LLMs effectively characterize software supply chain failures when the source articles are detailed enough for consensus among manual analysts, but cannot yet replace human analysts. Future work can improve LLM performance in this context, and study a broader range of articles and failures.

Distinguishing AI- and Human-Generated Code: A Case Study
  • Sufiyan Bukhari
  • Benjamin Tan
  • Lorenzo De Carli

While the use of AI assistants for code generation has the potential to revolutionize the way software is produced, assistants may generate insecure code, either by accident or as a result of poisoning attacks. They may also inadvertently violate copyright laws by mimicking code protected by restrictive licenses. We argue for the importance of tracking the provenance of AI-generated code in the software supply chain, so that adequate controls can be put in place to mitigate risks. For that, it is necessary to have techniques that can distinguish between human- and AI-generate code, and we conduct a case study in regards to whether such techniques can reliably work. We evaluate the effectiveness of lexical and syntactic features for distinguishing AI- and human-generated code on a standardized task. Results show accuracy up to 92%, suggesting that the problem deserves further investigation.

SESSION: Technical Session: Policies & Enforcement

What Does it Look Like to Code-sign for an Entire Packaging Ecosystem?
  • William Woodruff

Code-signing is a critical part of the software supply chain story, and is consistently identified by both individual developers and package index maintainers as a high-demand, high-impact improvement to Open Source packaging ecosystems

Despite this, relatively few Open Source packaging ecosystems support code-signing at all, much less require it for all packages. Correspondingly, we identify two questions: (1) Why do so few packaging ecosystems support code-signing, much less require universal code-signing? (2) What technical, logistical, and social problems currently prevent the adoption of universal code-signing by major packaging ecosystems?

In this security-in-practice talk, we attempt to answer these questions from the meat-and-potatoes perspective of a (hypothetical) package index maintainer and contributor. Using the Python Package Index [5] as an example, we walk through the challenges of lifting a large and critical ecosystem into an improved security stance, describe the blockers and qualifications on doing so, and enumerate various ideal and practical outcomes for integrations with frameworks like Sigstore [1], in-toto [3], and TUF [2]. We conclude with a theoretical roadmap for universal code-signing as well as a more practical roadmap for already extant ecosystems.

The audience will be exposed to (or refamiliarized with) the history of code-signing attempts in various major Open Source packaging ecosystems. We will present a survey of strategies that have historically worked or failed, and will produce a "taxonomy" of technical, logistical, and human problems that an universal codesigning scheme must overcome.

Macaron: A Logic-based Framework for Software Supply Chain Security Assurance
  • Behnaz Hassanshahi
  • Trong Nhan Mai
  • Alistair Michael
  • Benjamin Selwyn-Smith
  • Sophie Bates
  • Padmanabhan Krishnan

Many software supply chain attacks exploit the fact that what is in a source code repository may not match the artifact that is actually deployed in one's system. This paper describes a logic-based framework that analyzes a software component and its dependencies to determine if they are built in a trustworthy fashion. The properties that are checked include the availability of build provenances and whether the build and deployment process of an artifact is tamper resistant. These properties are based on the open-source community efforts, such as SLSA, that enable an incremental approach to improve supply chain security. We evaluate our tool on the top-30 Java, Python, and npm open-source projects and show that the majority still do not produce provenances. Our evaluation also shows that a large number of open-source Java and Python projects do not have a transparent build platform to produce artifacts, which is a necessary requirement to increase the trust in the published artifacts. We show that our tool fills a gap in the current software supply chain security landscape, and by making it publicly available the open-source community can both benefit from and contribute to it.

Scalable Policies for Supply Chain Security
  • Thomas Hennen

Most of the discussion around how to apply policies in the domain of Software Supply Chain Security is focused on how to meet the policy goals of a specific project. The solutions often assume the project owners are highly motivated and can invest significant time in writing the detailed rules that ensure their policy goals are met. While this may be true of people interested in software supply chain security today, we must also develop solutions that are easy to adopt by project owners who are less interested in security and who may see new requirements as a roadblock to achieving the goals of their project. In addition, the solutions must still meet a high security bar and be easy to reason about across the entire industry's software development lifecycle (SDLC)

To address these problems we will discuss: · The differences between organizational, project, workflow, and time-of-use policies. · Why universal policies (whether for vulnerability management, SLSA adoption, etc. . .) are insufficient and project specific policies are needed. E.g. Universal policies - Can only meet the lowest common denominator across an organization - Do not allow for incremental adoption - Do not offer sufficient protection against substitution attacks · How to summarize the organization-wide risk posture that results from these policies.

SESSION: Technical Session: Risk Evaluation & Detection

Differential Static Analysis for Detecting Malicious Updates to Open Source Packages
  • Fabian Niklas Froh
  • Matías Federico Gobbi
  • Johannes Kinder

Modern software applications routinely integrate many third-party open source dependencies, with package managers delivering timely updates of the entire dependency tree. The downside is that malicious actors can inject malicious code into widely-used software packages, which is then distributed to potentially thousands of direct or indirect client applications. Such attacks on the software supply chain are no longer just theoretical curiosities, but a practical risk. To mitigate this risk, we propose a new approach using differential static analysis to flag malicious code modifications in package updates. We use specifications in the CodeQL query language to match suspicious behavior and compare results across package versions. Where we detect an anomalous change in behavior, we classify that package update as potentially malicious and requiring further analysis. We show that our approach successfully identifies all malicious versions on a dataset of packages with a history of malicious code; on a dataset of popular benign packages from the npm repository, we obtain on average 1.3% false alarms, demonstrating that our approach holds promise for practical deployment as a warning system on the open source software supply chain.

Estimating Security Risk Through Repository Mining
  • Tamas K. Lengyel

As open-source projects are becoming ever more present throughout the modern computing stack, it is crucial to be able to quickly spot problematic, poorly maintained, or otherwise "risky" projects. The OSSF Scorecard has been introduced to address this need for fast security risk assessment by mining software repository metadata. We can reasonably expect a project with a CI system, code-reviews, and lots of users to be less risky than a project without those qualities. Since measuring security risk directly is difficult, we test a proxy hypothesis that these factors should also correlate with a reduction of bugs in software and test our hypothesis by scanning thousands of popular C & C++ projects with static analysis tools.

(Nothing But) Many Eyes Make All Bugs Shallow
  • Elizabeth Wyss
  • Lorenzo De Carli
  • Drew Davidson

Open source package repositories have become a crucial component of the modern software supply chain since they enable developers to easily and rapidly import code written by others. However, low quality, poorly vetted code residing in such repositories exposes developers and end-users to dangerous bugs and vulnerabilities at a large scale. Such issues have recently led to the creation of government-backed verification standards pertaining to packages, as well as a significant body of developer folklore regarding what constitutes a reliable package. However, there exists little academic research assessing the relationships between recommended development practices and known package issues in this domain. Motivated by this gap in understanding, we conduct a large-scale study that formally evaluates whether adherence to these guidelines meaningfully impacts reported issues and bug maintenance activity across the most widely utilized npm packages (encompassing 7,162 packages with over 100K weekly downloads each), which unveiled wide disparities across package-level metrics. We find that it is only recommendations pertaining to a broad notion of scrutiny that provide strong and reliable insights into the reporting and resolving of package issues. These findings pose significant implications for developers, who seek to identify well-maintained packages for use, as well as security researchers, who seek to identify suspicious packages for critical observation.

The Hitchhiker's Guide to Malicious Third-Party Dependencies
  • Piergiorgio Ladisa
  • Merve Sahin
  • Serena Elisa Ponta
  • Marco Rosa
  • Matias Martinez
  • Olivier Barais

The increasing popularity of certain programming languages has spurred the creation of ecosystem-specific package repositories and package managers. Such repositories (e.g., npm, PyPI) serve as public databases that users can query to retrieve packages for various functionalities, whereas package managers automatically handle dependency resolution and package installation on the client side. These mechanisms enhance software modularization and accelerate implementation. However, they have become a target for malicious actors seeking to propagate malware on a large scale. In this work, we show how attackers can leverage capabilities of popular package managers and languages to achieve arbitrary code execution on victim machines, thereby realizing open-source software supply chain attacks. Based on the analysis of 7 ecosystems, we identify 3 install-time and 4 runtime techniques, and we provide recommendations describing how to reduce the risk when consuming third-party dependencies. We provide example implementations that demonstrate the identified techniques. Furthermore, we describe evasion strategies employed by attackers to circumvent detection mechanisms.

SESSION: Technical Session: SBOMs

Software Bill of Materials in Java
  • Musard Balliu
  • Benoit Baudry
  • Sofia Bobadilla
  • Mathias Ekstedt
  • Martin Monperrus
  • Javier Ron
  • Aman Sharma
  • Gabriel Skoglund
  • César Soto-Valero
  • Martin Wittlinger

Modern software applications are virtually never built entirely in-house. As a matter of fact, they reuse many third-party dependencies, which form the core of their software supply chain [1]. The large number of dependencies in an application has turned into a major challenge for both security and reliability. For example, to compromise a high-value application, malicious actors can choose to attack a less well-guarded dependency of the project [2]. Even when there is no malicious intent, bugs can propagate through the software supply chain and cause breakages in applications. Gathering accurate, upto- date information about all dependencies included in an application is, therefore, of vital importance.

Enforcing SBOMs through the Linux kernel with eBPF and IMA
  • Alex Crawford
  • Eugene Yakubovich
  • Rob Szumski

At build-time, we generate SBOMs and sign containers. But at run-time, tools like Open Policy Agent and Kyverno only check the signatures or Kubernetes Pod attributes, not the full contents. Let's go beyond only checking container signatures and explore the possibility of checking an entire container against its software bill of materials (SBOM) in real time. This paper will discuss how features in the Linux kernel - Integrity Measurement Architecture (IMA) and eBPF - could eventually be used on a cluster's nodes to secure running containers and potentially other software. We are aiming to provide this protection without requiring a reboot or installing non-standard kernel modules. This would allow end users to protect machines via a Kubernetes DaemonSet or systemd unit without a need to tweak an operating system image or use a custom kernel.