October 6, 2022

Archives for May 2018

Princeton Dialogues of AI and Ethics: Launching case studies

Summary: We are releasing four case studies on AI and ethics, as part of the Princeton Dialogues on AI and Ethics.

The impacts of rapid developments in artificial intelligence (“AI”) on society—both real and not yet realized—raise deep and pressing questions about our philosophical ideals and institutional arrangements. AI is currently applied in a wide range of fields—such as medical diagnosis, criminal sentencing, online content moderation, and public resource management—but it is only just beginning to realize its potential to influence practically all areas of human life, including geopolitical power balances. As these technologies advance and increasingly come to mediate our everyday lives, it becomes necessary to consider how they may reflect prevailing philosophical perspectives and preferences. We must also assess how the architectural design of AI technologies today might influence human values in the future. This step is essential in order to identify the positive opportunities presented by AI and unleash these technologies’ capabilities in the most socially advantageous way possible while being mindful of potential harms. Critics question the extent to which individual engineers and proprietors of AI should take responsibility for the direction of these developments, or whether centralized policies are needed to steer growth and incentives in the right direction. What even is the right direction? How can it be best achieved?

Princeton’s University Center for Human Values (UCHV) and the Center for Information Technology Policy (CITP) are excited to announce a joint research project, “The Princeton Dialogues on AI and Ethics,” in the emerging field of artificial intelligence (broadly defined) and its interaction with ethics and political theory. The aim of this project is to develop a set of intellectual reasoning tools to guide practitioners and policy makers, both current and future, in developing the ethical frameworks that will ultimately underpin their technical and legislative decisions. More than ever before, individual-level engineering choices are poised to impact the course of our societies and human values. And yet there have been limited opportunities for AI technology actors, academics, and policy makers to come together to discuss these outcomes and their broader social implications in a systematic fashion. This project aims to provide such opportunities for interdisciplinary discussion, as well as in-depth reflection.

We convened two invitation-only workshops in October 2017 and March 2018, in which philosophers, political theorists, and machine learning experts met to assess several real-world case studies that elucidate common ethical dilemmas in the field of AI. The aim of these workshops was to facilitate a collaborative learning experience which enabled participants to dive deeply into the ethical considerations that ought to guide decision-making at the engineering level and highlight the social shifts they may be affecting. The first outcomes of these deliberations have now been published in the form of case studies. To access these educational materials, please see our dedicated website https://aiethics.princeton.edu. These cases are intended for use across university departments and in corporate training in order to equip the next generation of engineers, managers, lawyers, and policy makers with a common set of reasoning tools for working on AI governance and development.

In March 2018, we also hosted a public conference, titled “AI & Ethics,” where interested academics, policy makers, civil society advocates, and private sector representatives from diverse fields came to Princeton to discuss topics related to the development and governance of AI: “International Dimensions of AI” and “AI and Its Democratic Frontiers”. This conference sought to use the ethics and engineering knowledge foundations developed through the initial case studies to inspire discussion on AI technology’s wider social effects.

This project is part of a wider effort at Princeton University to investigate the intersection between AI technology, politics, and philosophy. There is a particular emphasis on the ways in which the interconnected forces of technology and its governance simultaneously influence and are influenced by the broader social structures in which they are situated. The Princeton Dialogues on AI and Ethics makes use of the university’s exceptional strengths in computer science, public policy, and philosophy. The project also seeks opportunities for cooperation with existing projects in and outside of academia.

How to constructively review a research paper

Any piece of research can be evaluated on three axes:

  • Correctness/validity — are the claims justified by evidence?
  • Impact/significance — how will the findings affect the research field (and the world)?
  • Novelty/originality — how big a leap are the ideas, especially the methods, compared to what was already known?

There are additional considerations such as the clarity of the presentation and appropriate citations of prior work, but in this post I’ll focus on the three primary criteria above. How should reviewers weigh these three components relative to each other? There’s no single right answer, but I’ll lay out some suggestions.

First, note that the three criteria differ greatly in terms of reviewers’ ability to judge them:

  • Correctness can be evaluated at review time, at least in principle.
  • Impact can at best be predicted at review time. In retrospect (say, 10 years after publication), informed peers will probably agree with each other about a paper’s impact.
  • Novelty, in contrast to the other two criteria, seems to be a fundamentally subjective notion.

We can all agree that incorrect papers should not be accepted. Peer review would lose its meaning without that requirement. In practice, there are complications ranging from the difficulty of verifying mathematical proofs to the statistical nature of research claims; the latter has led to replication crises in many fields. But as a principle, it’s clear that reviewers shouldn’t compromise on correctness.

Should reviewers even care about impact or novelty?

It’s less obvious why peer review should uphold standards of (predicted) impact or (perceived) novelty. If papers weren’t filtered for impact, presumably it would burden readers by making it harder to figure out which papers to pay attention to. So peer reviewers perform a service to readers by rejecting low-impact papers, but this type of gatekeeping does collateral damage: many world-changing discoveries were initially rejected as insignificant.

The argument for novelty of ideas and methods as a review criterion is different: we want to encourage papers that make contributions beyond their immediate findings, that is, papers that introduce methods that will allow other researchers to make new discoveries in the future.

In practice, novelty is often a euphemism for cleverness, which is a perversion of the intent. Readers aren’t served by needlessly clever papers. Who cares about cleverness? People who are evaluating researchers: hiring and promotion committees. Thus, publishing in a venue that emphasizes novelty becomes a badge of merit for researchers to highlight in their CVs. In turn, forums that publish such papers are seen as prestigious.

Because of this self-serving aspect, today’s peer review over-emphasizes novelty. Sure, we need occasional breakthroughs, but mostly science progresses in a careful, methodical way, and papers that do this important work are undervalued. In many fields of study, publishing is at risk of devolving into a contest where academics impress each other with their cleverness.

There is at least one prominent journal, PLoS One, whose peer reviewers are tasked with checking only correctness, with impact and novelty being left to be sorted out post-publication. But for most journals and peer-reviewed conferences, the limited number of publication slots means that there will inevitably be gatekeeping based on impact and/or novelty.

Suggestions for reviewers

Given this reality, here are four suggestions for reviewers. This list is far from comprehensive, and narrowly focused on the question of weighing the three criteria.

  1. Be explicit about how you rate the paper on correctness, impact, and novelty (and any other factors such as clarity of the writing). Ideally, review forms should insist on separate ratings for the criteria. This makes your review much more actionable for the authors: should they address flaws in the work, try harder to convince the world of its importance, or abandon it entirely?
  2. Learn to recognize your own biases in assessing impact and novelty, and accept that these assessments might be wrong or subjective. Be open to a discussion with other reviewers that might change your mind.
  3. Not every paper needs to maximize all three criteria. Consider accepting papers with important results even if they aren’t highly novel, and conversely, papers that are judged to be innovative even if the potential impact isn’t immediately clear. But don’t reward cleverness for the sake of cleverness; that’s not what novelty is supposed to be about.
  4. Above all, be supportive of authors. If you rated a paper low on impact or novelty, do your best to explain why.

Conclusion

Over the last 150 years, peer review has evolved to be more and more of a competition. There are some advantages to this model, but it makes it easy for reviewers to lose touch with the purpose of peer review and basic norms of civility. Once in a while, we need to ask ourselves critical questions about what we’re doing and how best to do it. I hope this post was useful for such a reflection.

 

Thanks to Ed Felten and Marshini Chetty for feedback on a draft.

 

When Terms of Service limit disclosure of affiliate marketing

By Arunesh Mathur, Arvind Narayanan and Marshini Chetty

In a recent paper, we analyzed affiliate marketing on YouTube and Pinterest. We found that on both platforms, only about 10% of all content with affiliate links is disclosed to users as required by the FTC’s endorsement guidelines.

One way to improve the situation is for affiliate marketing companies (and other “influencer” agencies) to hold their registered content creators to the FTC’s endorsement guidelines. To better understand affiliate marketing companies’ current practices, we examined the terms and conditions of eleven of the most common affiliate marketing companies in our dataset, and specifically noted whether they required content creators to disclose their affiliate content or whether they mentioned the FTC’s guidelines upon registration.

Affiliate program Requires disclosure?
AliExpress No
Amazon Yes
Apple No
Commission Junction No
Ebay Yes
Impact Radius No
Rakuten Marketing No
RewardStyle N/A
ShopStyle Yes
ShareASale No

The table above summarizes our findings. All the terms and conditions were accessed May 1, 2018 from the affiliate marketing companies’ websites. We did not hyperlink those terms and conditions that were not available publicly. All the companies that required disclosure also mentioned the FTC’s endorsement guidelines.

Out of the top 10 programs in our corpus, only 3 explicitly instructed their creators to disclose their affiliate links to their users. In all three cases (Amazon, Ebay, and ShopStyle), the companies called out the FTC’s endorsement guidelines. Of particular interest is Amazon’s affiliate marketing terms and conditions (Amazon was the largest affiliate marketing program in our dataset).

Amazon’s terms and conditions: When content creators sign up on Amazon’s website, they are bound by the programs terms and agreements Section 5 titled: “Identifying Yourself as an Associate”.

Figure 1: The disclosure requirement in Section 5 of Amazon’s terms and conditions document.

As seen in Figure 1, the terms of Section 5 do not explicitly mention the FTC’s endorsement guidelines but constrain participants to add only the following disclosure to their content: “As an Amazon Associate I earn from qualifying purchases”. In fact, the terms go so far as to warn users that “Except for this disclosure, you will not make any public communication with respect to this Agreement or your participation in the Associates Program”.

However, if participants click on the “Program Policies” link in the terms and conditions—which they are also bound to by virtue of agreeing to the terms and conditions—they are specifically asked to be responsible for the FTC’s endorsement guidelines (Figure 2): “For example, you will be solely responsible for… all applicable laws (including the US FTC Guides Concerning the Use of Endorsement and Testimonials in Advertising)…”. Here, Amazon asks the content creators to comply with the FTC’s guidelines, without exactly specifying how. It is important to note that the FTC’s guidelines themselves do not enforce any specific disclosure statement constraints on content creators, but rather suggest that content creators use clear and explanatory disclosures that convey the advertising relationship behind affiliate marketing to users.

Figure 2: The disclosure requirement from Amazon’s “Program Policies” page.

We learned about these clauses from the coverage of our paper on BBC’s You and Yours podcast (~ 16 mins in). A YouTuber on the show pointed out that he was constrained by the Amazon’s clause to not disclose anything about the affiliate program publicly.

Indeed, as we describe in the above sections, Amazon’s terms and conditions seem contradictory to their Program Policies. On the one hand, Amazon binds its participants to the FTC’s endorsement guidelines but on the other, Amazon severely constrains the disclosures content creators can make about their participation in the program.

Further, researchers are still figuring out which types of disclosures are effective from a user perspective. Content creators might want to adapt the form and content of disclosures based on the findings of such research and the affordances of the social platforms. For example, on YouTube, it might be best to call out the affiliate relationship in the video itself—when content creators urge participants to “check out the links in the description below”—rather than merely in the description. The rigid wording mandated by Amazon seemingly prevents such customization, and may not make the affiliate relationship adequately clear to users.

Affiliate marketing companies wield strong influence over the content creators that register with their programs, and can hold them accountable to ensure they disclose these advertising relationships in their content. At the very least, they should not make it harder to comply with applicable laws and regulations.

Refining the Concept of a Nutritional Label for Data and Models

By Julia Stoyanovich (Assistant Professor of Computer Science at Drexel University)  and Bill Howe (Associate Professor in the Information School at the University of Washington)

In August 2016,  Julia Stoyanovich and Ellen P. Goodman spoke in this forum about the importance of bringing interpretability to the algorithmic transparency debate.  They focused on algorithmic rankers, discussed the harms of opacity, and argued that the burden on making ranked outputs transparent rests with the producer of the ranking.   They went on to propose a “nutritional label” for rankings called Ranking Facts.

In this post, Julia Stoyanovich and Bill Howe discuss their recent technical progress on bringing the idea of Ranking Facts to life, placing the nutritional label metaphor in the broader context of the ongoing algorithmic accountability and transparency debate.

In 2016, we began with a specific type of nutritional label that focuses on algorithmic rankers.  We have since developed a Web-based Ranking Facts tool, which will be presented at the upcoming ACM SIGMOD 2018 conference.   

Figure 1: Ranking Facts on the CS departments datasetThe Ingredients widget (green) has been expanded to show the details of the attributes that strongly influence the ranking.  The Fairness widget (blue) has been expanded to show details of the fairness computation.

Figure 1 presents Ranking Facts for CS department rankings, the same dataset as was used for illustration in our August 2016 post.  The nutritional label was constructed automatically, and consists of a collection of visual widgets, each with an overview and a detailed view.  

  • Recipe widget succinctly describes the ranking algorithm. For example, for score-based ranker that uses a linear scoring formula to assign as score to each item, each attribute would be listed together with its weight.
  • Ingredients widget lists attributes most material to the ranked outcome, in order of importance. For example, for a linear model, this list could present the attributes with the highest learned weights.
  • Stability widget explains whether the ranking methodology is robust on this particular dataset – would small changes in the data, such as those due to uncertainty or noise, result in significant changes in the ranked order?  
  • Fairness and Diversity widgets quantify whether the ranked outcome exhibits parity (according to some measure – three such measures are presented in Figure 1), and whether the set of results is diverse with respect to one or several demographic characteristics.

What’s new about nutritional labels?

The database and cyberinfrastructure communities have been studying systems and standards for metadata, provenance, and transparency for decades.  For example, the First Provenance Challenge in 2008 led to the creation of the Open Provenance Model that standardized years of previous efforts across multiple communities,   We are now seeing renewed interest in these topics due to the proliferation of machine learning applications that use data opportunistically.  Several projects are emerging that explore this concept, including Dataset Nutrition Label at the Berkman Klein Center at Harvard & the MIT Media LabDatasheets for Datasets, and some emerging work about Data Statements for NLP datasets from Bender and Friedman.  In our work, we are interested in automating the creation of nutritional labels, for both datasets and models, and in providing open source tools for others to use in their projects.

Is a nutritional label simply an apt new name for an old idea?  We think not! We see nutritional labels as a unifying metaphor that is responsive to changes in how data is being used today.  

Datasets are now increasingly used to train models to make decisions once made by humans.  In these automated systems, biases in the data are propagated and amplified with no human in the loop.  The bias, and the effect of the bias on the quality of decisions made, is not easily detectable due to the relative opacity of the system.  As we have seen time and time again, models will appear to work well, but will silently and dangerously reinforce discrimination. Worse, these models will legitimize the bias — “the computer said so.”  So we are designing nutritional labels for data and models to respond specifically to the harms implied by these scenarios, in contrast to the more general concept of just “data about data.”

Use cases for nutritional labels: Enhancing data sharing in the public sector

Since we first began discussing nutritional labels in 2016, we’ve seen increased interest from  the public sector in scenarios where data sharing is considered high-risk. Nutritional labels can be used to support data sharing, while mitigating some of the associated risks. Consider these examples:

Algorithmic transparency law in New York City

New York City recently passed a law requiring that a task force be put in place to survey the current use of “automated decision systems,” defined as “computerized implementations of algorithms, including those derived from machine learning or other data processing or artificial intelligence techniques, which are used to make or assist in making decisions,” in City agencies.  The task force will develop a set of recommendations for enacting algorithmic transparency, which, as we argued in our testimony before the New York City Council Committee on Technology regarding Automated Processing of Data, cannot be achieved without data transparency. Nutritional labels can support data transparency and interpretability,  surfacing the statistical properties of a dataset, the methodology that was used to produce it, and, ultimately, substantiating the “fitness for use” of a dataset in the context of a specific automated decision system or task.

Addressing the opioid epidemic

An effective response to the opioid epidemic requires coordination between at least three sectors: health care, criminal justice, and emergency housing.  An optimization problem is to effectively, fairly and transparently assign resources, such as hospital rooms, jail cells, and shelter beds,  to at-risk citizens.  Yet, centralizing all data is disallowed by law, and solving the global optimization problem is therefore difficult. We’ve seen interest in nutritional labels to share the details of local resource allocation strategies, to help bootstrap a coordinated response without violating data sharing principles.  In this case the nutritional labels are shared separately from the datasets themselves.

Mitigating urban homelessness

With the Bill and Melinda Gates Foundation, we are integrating data about homeless families from multiple government agencies and non-profits to understand how different pathways through the network of services affect outcomes.  Ultimately, we are using machine learning to deliver prioritized recommendations to specific families. But the families and case workers need to understand how a particular recommendation was made, so they can in turn make an informed decision about whether to follow it.  For example, income levels, substance abuse issues, or health issues may all affect the recommendation, but only the families themselves know whether the information is reliable.

Sharing transportation data

At the University of Washington, we are developing the Transportation Data Collaborative, an honest broker system that can provide reports and research to policy makers while maintaining security and privacy for sensitive information about companies and individuals.  We are releasing nutritional labels for reports, models, and synthetic datasets that we produce to share known biases about the data and our methods of protecting privacy.

Properties of a nutritional label

To differentiate a nutritional label from more general forms of metadata, we articulate several properties:

  • Comprehensible: The label is not a complete (and therefore overwhelming) history of every processing step applied to produce the result.  This approach has its place and has been extensively studied in the literature on scientific workflows, but is unsuitable for the applications we target.  The information on a nutritional label must be short, simple, and clear.
  • Consultative: Nutritional labels should provide actionable information, rather than just descriptive metadata.  For example, universities may invest in research to improve their ranking, or consumers may cancel unused credit card accounts to improve their credit score.
  • Comparable: Nutritional labels enable comparisons between related products, implying a standard.
  • Concrete: The label must contain more than just general statements about the source of the data; such statements do not provide sufficient information to make technical decisions on whether or not to use the data.

Data and models are chained together into complex automated pipelines — computational systems “consume” datasets at least as often as people do, and therefore also require nutritional labels!  We articulate additional properties in this context:

  • Computable: Although primarily intended for human consumption, nutritional labels should be machine-readable to enable specific applications: data discovery, integration, automated warnings of potential misuse.  
  • Composable: Datasets are frequently integrated to construct training data; the nutritional labels must be similarly integratable.  In some situations, the composed label is simple to construct: the union of sources. In other cases, the biases may interact in complex ways: a group may be sufficiently represented in each source dataset, but underrepresented in their join.  
  • Concomitant: The label should be carried with the dataset; systems should be designed to propagate labels through processing steps, modifying the label as appropriate, and implementing the paradigm of transparency by design.

Going forward

We are interested in the application of nutritional labels at various stages in the data science lifecycle: Data scientists triage datasets for use to train their models; data practitioners inspect and validate trained models before deploying them in their domains; consumers review nutritional labels to understand how decisions that affect them were made and how to respond.  

The software infrastructure implied by nutritional labels suggests a number of open questions for the computer science community: Under what circumstances can nutritional labels be generated automatically for a given dataset or model? Can we automatically detect and report potential misuse of datasets or models, given the information in a nutritional label?  We’ve suggested that nutritional labels should be computable, composable, and concomitant — carried with the datasets to which they pertain; how can we design systems that accommodate these requirements?  

We look forward to opening these discussions with the database community at two upcoming events:  at ACM SIGMOD 2018, where we are organizing a special session on a technical research agenda in data ethics and responsible data management,  and at VLDB 2018, where we will run a debate on data and algorithmic ethics.