"Deep learning" is the name given to a branch of machine learning that studies and applies large, complex neural networks to a range of learning problems. The word "deep" comes from the large depth (number of layers) of the neural networks used, which differentiates it from early work using neural networks. An additional characteristic of the field is that typically the performance of a system is dependent on having large amounts of data available for training, which is possible in more and more domains as computers, digitization, and information technology permeate more and more aspects of society.
Deep learning is an area of increased interest now because of substantial advances and breakthroughs in the field over the past decade (starting around 2011-2012). Using deep learning techniques, researchers have produced systems that approach or sometimes exceed human-level performance on a range of tasks that were previously difficult for computers (given the previously best known algorithms). These tasks include everything from image recognition (and computer vision more broadly) to text generation (and natural language processing more broadly) to playing Go. In these areas and more, deep learning has produced substantial improvements in a short time, which has caught the interest of everyone from academic researchers to CEOs as they consider whether it can be used to solve their problems of interest. The rapid rise of the field and the nature of many of its applications raise a number of important ethical concerns as well.
This class will study deep learning from several perspectives. First, we will learn the mathematical foundations of building and using artificial neural networks and their implementation in software. With that base of understanding, we will build increasingly complex neural network structures to solve a variety of problems, studying their implementation and limitations along the way. We will study several specific deep learning network structures that have been found to perform well in a domains, including convolutional neural networks for computer vision and transformers for natural language processing. And along the way, we will study and consider the ethical implications and issues raised by the use of these complex, often inscrutable systems throughout society.
For an idea of the specific topics covered in the course, see the rough schedule for the semester.
Semester schedule — tentative - see Canvas for up-to-date details.
Online Resources — notes, readings, and other resources will be posted here. I will link to relevant entries from Canvas, but it doesn't hurt to browse and see what else is there.
The final grade will be based roughly on the following breakdown:
| Assignments | 20% |
| Quizzes | 10% |
| Exams | 30% |
| Project | 40% |
Assignments will be posted on Canvas, usually about a week before they are due, and they will be collected there as well.
Quizzes will be short and frequent. Each will cover a small set of recent topics and/or the readings and pre-class work for a class period. Quizzes will be administered via Canvas.
The project will be a coding project in which you develop a substantial program that involves some aspect of deep learning. These projects may be done individually or in groups of two. Groups will be expected to tackle correspondingly larger problems than individuals. For the first half of the semester, explore potential topics and gather ideas for your project. Feel free to discuss ideas with me as you have them. I will also provide ideas for projects. Part way through the semester, everyone will schedule a meeting with me to discuss ideas and choose one.
Several documents will be written during the project:
Detailed descriptions of each of these documents will be provided on Canvas.
During our final period, we will have a presentation session. Every project will get a presentation slot, in which you will present the project to the class. We may hold some presentations earlier if needed.
The final grade for the project will be based on the initial proposal, progress reports, the final report (including an assessment of the code itself), and the final presentation.
Assignments will be due at set times; they will be considered late at any point after that time. An assignment will lose 10% of the total possible points for every day or partial day it is late, and after five days it will not be accepted.
Assignments can't be accepted at all after solutions have been handed out or the graded work has been returned to the class.
Every student has two "grace tokens" that they may use for extensions in instances where they are unable to complete work by the assigned deadline. To use a grace token on an assignment, send me an email before the assignment deadline, explain why you need an extension, and we will determine an appropriate extension, which will be granted with no grade penalty. Some assignments may not be eligible for grace tokens due to immediate use of or feedback on the submitted work, but most will be.
There will be two exams during the semester, held in class.
If you would like to request a regrade, submit a request in writing (via email) within one week of receiving the graded assignment, exam, etc. Indicate exactly which part you believe deserves a different score and why.
If any concept, piece of code, compiler error, or anything else related to this course is ever unclear to you, please come see me during my office hours. One-on-one, I can help you bridge the gap from what you do understand to what you want to understand. It will generally be the easiest, fastest way to clarify something.
If you cannot attend any of my regularly scheduled office hours, I'm more than happy to setup another time to meet. Just let me know.
And to make sure that misunderstandings and difficulties do not go unaddressed, I will often ask that anyone who receives a C- or below on any assignment or exam come see me to see what we can do about improving that going forward.
I strongly encourage you to form study groups with your classmates, compare notes, explain concepts to one another, and generally help each other learn the material in this course.
Any material turned in for a grade must be your own individual work, though. You may work on concepts with other students, but I ask that you not discuss assigned problems until after the work has been turned in. Giving or showing your solutions or code to another student — or looking at theirs — is not allowed.
This has two goals: 1) ensure that grades are a reflection of each student's own work, and 2) avoid situations where one person solves a problem and another records the answer as their own work without really learning.
Submitting solutions generated by a large language model (AI) as your own work is plagiarism. Copying code found online and submitting it as your own work is plagiarism as well. Copied code with a source cited will not be reported as an academic integrity violation, but if the code you've copied is a substantial portion of the problem you were supposed to solve, you may still lose points.
For details on the university's policies regarding academic integrity, please read the University Policies Concerning Student Conduct & Academic Integrity. Academic misconduct can result in failing the course and a report to the associate provost. If you are ever unsure of whether something might be crossing that line, please err on the side of caution. You can also just ask me, and I'll be happy to provide guidance.
Illinois Wesleyan University strives to make all learning experiences as accessible as possible. If you anticipate or experience academic barriers based on a disability (including mental health and chronic or temporary medical conditions), it is your responsibility to self-disclose and provide documentation to the Office of Student Accessibility Services. Please note that accommodations are not retroactive and accommodations cannot be provided until I receive an email from Student Accessibility Services. Once the email is sent, please make arrangements with me as soon as possible to discuss your accommodations confidentially so they may be implemented in a timely fashion. For more information contact Student Accessibility Services by e-mailing accessibility@iwu.edu or stopping by their office in Holmes Hall.
Our university's mission statement includes, "The University through our policies, programs and practices is committed to diversity [...]" Our school and this course are made stronger by the mix of people that come into it bringing a diversity of ideas, experiences, and backgrounds. I expect everyone in this course — instructor, TA, and student — to contribute to an inclusive atmosphere that respects the diversity of all others in it. Dimensions of diversity can include sex, race, age, national origin, ethnicity, gender identity and expression, intellectual and physical ability, sexual orientation, income, faith and non-faith perspectives, socio-economic class, political ideology, education, primary language, family status, military experience, cognitive style, and communication style. The individual intersection of these experiences and characteristics must be valued in our community. If you have related concerns about the class environment or behavior of any in it (including me), you are welcome to raise them with me, and I will do my best to address them. If you are not comfortable speaking with me about them, you may also bring concerns to the Associate Provost or the Office of Diversity and Inclusion.
Experiment. The assignments in this course will guide you through implementing various concepts to help you better understand them and their implementation. It would be wise to go beyond the path laid out by each assignment and explore further. Maybe you don't understand regularization as well as you'd like, so you write a little bit of extra code to stretch the bounds of what the relevant assignment has you do with it to gain some more insight. Maybe you have an idea about an alternative approach to something that might work, so you try it out to see what it does. There is a lot of room in this course to take what you're given and expand it in various ways, experimenting with code to better understand it, explore ideas, and generally learn things better. I realize this is easier said than done given time pressures students are under and the need to prioritize, but it would be wise. I can give a bit of extra credit in the "Engagement" portion of the grade if you do something interesting beyond what the assignments require — after all, that would be a great way to engage with the material.