LING 575 — Ethical Considerations in NLP

A quick pitch for my upcoming WIN 19 course, covering some of the topics from this course.

N.B.: If you are a student enrolled in this course, please refer to the course page on Canvas. This page is for public-facing and archival purposes and may not be up-to-date for coursework.

Course Info

  • Lecture: Wednesdays, 3:30-5:50 in MGH 288 and online

Instructor Info

  • Ryan Georgi
  • Office Hours: Wednesdays 12:30-2:30.
  • Office: GUG 418-D

Description

The goal of this course is to better understand the ethical considerations that arise in the deployment of NLP technology, including (but not limited to) considerations of demographic misrepresentation, bias confirmation, and privacy. We will start with foundations in ethics, and then move to the current and growing research literature on ethics in NLP and allied fields, before considering specific NLP tasks, data sets and training methodologies through the lens of the ethical considerations identified. Course projects are expected to take the form of a term paper analyzing some particular NLP task or data set in terms of the concepts developed through the quarter and looking forward to how ethical best practices could be developed for that task/data set.

In particular, I hope to find answers to the following guiding questions over the course of the term:

  • What ethical considerations arise in the design and deployment of NLP technologies?
  • Which of these are specific to NLP (as opposed to AI or technology more generally?)
  • What best practices can/should NLP developers deploy in light of the ethical concerns identified?

Note: To request academic accommodations due to a disability, please contact Disability Resources for Students , 448 Schmitz, 206-543-8924 (V/TTY). If you have a letter from DSR indicating that you have a disability which requires academic accommodations, please present the letter to the instructor so we can discuss the accommodations you might need in this class.

Requirements

Schedule of Topics and Assignments (subject to change)

Date Topic Reading Due
3/28 Introduction, organization
Why are we here? What do we hope to accomplish?
Hovy and Spruit 2016 plus at least 2 other papers/articles listed under Overviews/Calls to Action (or just one, if you pick something particularly long)
4/2 KWLA papers: K & W due 11pm
4/4 Philosophical foundations 2 items from Philosophical Foundations, at least one of which comes from an author whose perspective varies greatly from your own life experience. Be prepared to discuss the following:

  • What is the main thesis of the reading?
  • What is their definition of ethics?
  • In what ways do they contrast their definition with others?
  • How does this reading relate to ethics in NLP?
4/11 Philosophical foundations (cont)
4/18 Exclusion/Discrimination/Bias 3–4 items from Exclusion/Discrimination/Bias, considering the following reading questions (not all of which are necessarily appropriate for all readings):

  • What went wrong?
  • Who was harmed?
  • Who benefitted?
  • What (if anything) is offered as a way to mitigate such harm in the future?
  • How does the reading you did for “philosophical foundations” relate to this issue?
  • What (if any) analogies do you see to the kind of NLP tasks you work on?
4/25 Word Embeddings and Language Behavior as Ground Truth
Chat bots
2 items from each of Word Embeddings and Language Behavior as Ground Truth and Chat bots, considering the following reading questions (not all of which are necessarily appropriate for all readings):

  • How do the word embedding readings relate to the distributional hypothesis? (“You know a word by the company it keeps”)
  • How do the scenarios described relate to the issue of using descriptive models prescriptively?
  • How does intent (user intent, system designer intent) relate to language generation tools?
  • In what ways do we find tension between user satisfaction and potential ethical considerations?
  • In what ways are chat bots beneficial?
  • What are the implications of gendering virtual assistants?
5/2 Proposed code of ethics for ACL
Term project brainstorm
Details

5/7 Term paper proposals due
5/9 Value Sensitive Design Read any two other papers from Value Sensitive Design. Reading questions:

  • How could you apply VSD theoretical constructs and methods to the NLP tasks you are most concerned with? Prepare two or three concrete examples.
  • How do VSD theoretical constructs and methods build on or provide counterpoint to what you read in Philosophical Underpinnings?

In addition, for an NLP project you are interested in:

  • Make a list of the direct and indirect stakeholders. Identify how each stakeholder group you identify might benefit or be harmed by the technology you are considering.
  • For those who choose the paper by Nathan et al. on value scenarios, write a value scenario like those illustrated in the paper for the technology you are interested in investigating.
5/14 Proposed NLP/ML ethics code critique due
5/16 Other Best Practices Read at least three papers from Other Best Practices. Reading/discussion questions:

  • What is shared with value sensitive design?
  • What contrasts to value sensistive design?
  • How could this be applied to [insert your favorite NLP task]?
Term paper outline due
5/23 Privacy Read at least three papers from Privacy. At least one should be from a CS-type perspective and at least one from a non-CS scholarly perspective (social sciences or law). Reading/discussion questions:

  • How are people addressing privacy; which ethical frameworks?
  • How is privacy defined?
  • What is privacy in tension with?
  • What purpose does privacy serve/why is it valued?
  • How has the notion of privacy changed over the last few decades?
  • What unique concerns are there in NLP and privacy?
5/28 Term paper draft due
5/30 NLP Applications Addressing Ethical Issues Choose three of the items under NLP Apps Addressing Ethical Issues below and be prepared to discuss the following reading questions:

  • What was the social issue addressed?
  • How well did it work/how could you carry out an evaluation if one wasn’t done?
  • Design noir: What could go wrong?
6/1 KWLA papers due
Comments on partner’s paper draft due
6/6 Final papers due 11pm

Bibliography

  1. Overviews/Calls to Action
  2. Philosophical Underpinnings
  3. Human Subjects & Social Media Research
  4. Exclusion/Representation/Discrimination/Bias
  5. Word Embeddings and Language Behavior as Ground Truth
  6. Chat Bots
  7. Abusive Language Online
  8. Privacy
  9. NLP Applications
  10. Crowdsourcing
  11. Other
  12. Value Sensitive Design
  13. Proposals for Codes of Ethics
  14. Other Best Practices
  15. Workshops
  16. Other Resources
  17. Other Courses

Overviews/Calls to Action

[Back to Top]

Philosophical Underpinnings

[Back to Top]

Human Subjects & Social Media Research

Exclusion/Representation/Discrimination/Bias

[Back to Top]

Word Embeddings and Language Behavior as Ground Truth

[Back to Top]

Chat bots

[Back to Top]

Abusive Language Online

Privacy

[Back to Top]

NLP Apps Addressing Ethical Issues

[Back to Top]

Crowdsourcing

[Back to Top]

Other

[Back to Top]

Value Sensitive Design

[Back to Top]

(Proposals for) codes of ethics

[Back to Top]

Other Best Practices

[Back to Top]

Links

Conferences/Workshops

[Back to Top]

Other lists of resources

[Back to Top]

Other courses

[Back to Top]