Why are these tools like this? - CSC311 - Computer Systems and Programming Tools

Discussion¶

share your responses to the work in the prep work for today’s topic
discuss any similiarities or differences
if any, send back questions you have
if any, send points you want to share to the whole class

How did the processes, decisions, and practices that people at your table brought up compare to what you thought of?

Design¶

In CS we tend to be more implicit about design, but that makes it hard to learn and keep track of.

Today’s goal is to be more explicit, disucssing some principles and seeing how you have already interacted wit them.

Today we’re going to do a bit more practical stuff, but we are also going to start to get into the philosophy of how things are organized.

Understanding the context and principles will help you:

remember how to do things
form reliable hypotheses about how to fix problems you have not seen before
help you understand when you should do things as they have always been done and when you should challenge and change things.

Why should we study design?¶

it is easy to get distracted by implementation, syntax, algorithms
but the core principles of design organize ideas into simpler rules

Why are we studying developer tools?¶

The best way to learn design is to study examples [Schon1984, Petre2016], and some of the best examples of software design come from the tools programmers use in their own work.

Software design by example

note

we will talk about some history in this course
I will not take a “great men of history” approach

This is because:

I think that history is important context for making decisions
Many of these “great men” are actually, in many ways, Not Great^TM
It is important to remember that all of this work was done by people
all people are imperfect
when people are deeply influential, ignoring their role in history is not effective, we cannot undo what they did
we do not have to admire them or even say their names to acknowledge the work
computing technology has been used in Very Bad ways and in Definitely Good ways

Unix Philosophy¶

sources:

wiki
a free book
composability over monolithic design
social conventions

The tenets:

Make it easy to write, test, and run programs.
Interactive use instead of batch processing.
Economy and elegance of design due to size constraints (“salvation through suffering”).
Self-supporting system: all Unix software is maintained under Unix.

For better or worse unix philosophy is dominant, so understanding it is valuable.

This critique is written that unix is not a good system for “normal folks” not in its effectiveness as a context for developers which is where *nix (unix, linux) systems remain popular.

context:
“normal folks” is the author’s term; my guess is that it is attempting to describe a typical, nondeveloper computer user terminology to refer to people has changed a lot over time; when we study concepts from primary sources, we have to interpret the document through the lens of what was normal at the time the document was written, not to excuse bad behavior but to not be distracted and see what is there

Today we will work in your main repo

cd fall25-brownsarahm/

Philosophy in practice, using pipes¶

To check if your gh CLI is working:

gh

  Work seamlessly with GitHub from the command line.

USAGE
  gh <command> <subcommand> [flags]

CORE COMMANDS
  auth:        Authenticate gh and git with GitHub
  browse:      Open the repository in the browser
  codespace:   Connect to and manage codespaces
  gist:        Manage gists
  issue:       Manage issues
  org:         Manage organizations
  pr:          Manage pull requests
  project:     Work with GitHub Projects.
  release:     Manage releases
  repo:        Manage repositories

GITHUB ACTIONS COMMANDS
  cache:       Manage GitHub Actions caches
  run:         View details about workflow runs
  workflow:    View details about GitHub Actions workflows

EXTENSION COMMANDS
  classroom:   Extension classroom

ALIAS COMMANDS
  co:          Alias for "pr checkout"

ADDITIONAL COMMANDS
  alias:       Create command shortcuts
  api:         Make an authenticated GitHub API request
  attestation: Work with artifact attestations
  completion:  Generate shell completion scripts
  config:      Manage configuration for gh
  extension:   Manage gh extensions
  gpg-key:     Manage GPG keys
  label:       Manage labels
  ruleset:     View info about repo rulesets
  search:      Search for repositories, issues, and pull requests
  secret:      Manage GitHub secrets
  ssh-key:     Manage SSH keys
  status:      Print information about relevant issues, pull requests, and notifications across repositories
  variable:    Manage GitHub Actions variables

HELP TOPICS
  actions:     Learn about working with GitHub Actions
  environment: Environment variables that can be used with gh
  exit-codes:  Exit codes used by gh
  formatting:  Formatting options for JSON data exported from gh
  mintty:      Information about using gh with MinTTY
  reference:   A comprehensive reference of all gh commands

FLAGS
  --help      Show help for command
  --version   Show gh version

EXAMPLES
  gh issue create
  gh repo clone cli/cli
  gh pr checkout 321

LEARN MORE
  Use `gh <command> <subcommand> --help` for more information about a command.
  Read the manual at https://cli.github.com/manual
  Learn about exit codes using `gh help exit-codes`

When we use it without a subcommand, it gives us help output a list of all the commeands that we can use. If it is not working, you would get a command not found error.

Let’s inpsect the action file that creates the issues for your badges.

cat .github/workflows/getassignment.yml

cat .github/workflows/getassignment.yml

.github/workflows/getassignment.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
name: Create badge issues (Do not run manually)
on:
  workflow_dispatch

jobs:
  check-contents:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    
    # Install dependencies
    - name: Set up Python 
      uses: actions/setup-python@v5
      with:
        python-version: 3.12
    
    - name: Install Utils
      run: |
        pip install git+https://github.com/compsys-progtools/courseutils@main
    - name: Get badge requirements
      run: |
        # prepare badge lines
        pretitle="prepare-"$(cspt getbadgedate --prepare)
        cspt getassignment --type prepare | gh issue create --title $pretitle --label prepare --body-file -
        # review badge lines
        rtitle="review-"$(cspt getbadgedate --review)
        cspt getassignment --type review | gh issue create --title $rtitle --label review --body-file -
        # practice badge lines
        pratitle="practice-"$(cspt getbadgedate --practice)
        cspt getassignment --type practice | gh issue create --title $pratitle --label practice --body-file -
      env:
        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
#  edit the run step above for the level(s) you want. 
#  You should keep the prepare, because they are required for experience badges 
#     You may choose to get only the review or only the practice (and change this any time) 
# added comment to fix

A lot of this is familiar, but we are focused today on the three highlighed lines

Here we see a few key things:

on the last line it uses a pipe | to connect a command sysgetassignment to the gh issue create command.
the gh issue create command uses the --body-file option with a value -; this uses std in, but since this is to the right of the pipe, it puts the output of the first command into this option
the 2nd to last line creates a variable (we will learn this more later) and the last line uses that variable
the Install Utils step, installs some custom code.

We know that this action is what is used to create badge issues, so this custom code must be what gets information from the course website to get the assignments and prepare them for your badge issues.

Installing from source¶

As we saw in the action, we can install python packages from source via git. let’s install the new version of the code

pip install git+https://github.com/compsys-progtools/courseutils@main

Collecting git+https://github.com/compsys-progtools/courseutils@main
  Cloning https://github.com/compsys-progtools/courseutils (to revision main) to /private/var/folders/8g/px8bm7bj0_j31j71yh6mfd_r0000gn/T/pip-req-build-gicd88lk
  Running command git clone --filter=blob:none --quiet https://github.com/compsys-progtools/courseutils /private/var/folders/8g/px8bm7bj0_j31j71yh6mfd_r0000gn/T/pip-req-build-gicd88lk
  Resolved https://github.com/compsys-progtools/courseutils to commit 395ac8754c55a119ffa2b3670afabfe4ba082c9e
  Preparing metadata (setup.py) ... done
Requirement already satisfied: Click in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from syscourseutils==1.0.6) (8.1.7)
Requirement already satisfied: pandas in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from syscourseutils==1.0.6) (2.2.2)
Requirement already satisfied: lxml in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from syscourseutils==1.0.6) (5.3.0)
Requirement already satisfied: pyyaml in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from syscourseutils==1.0.6) (6.0.2)
Requirement already satisfied: numpy in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from syscourseutils==1.0.6) (2.1.1)
Requirement already satisfied: requests in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from syscourseutils==1.0.6) (2.32.3)
Requirement already satisfied: html5lib in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from syscourseutils==1.0.6) (1.1)
Requirement already satisfied: six>=1.9 in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from html5lib->syscourseutils==1.0.6) (1.16.0)
Requirement already satisfied: webencodings in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from html5lib->syscourseutils==1.0.6) (0.5.1)
Requirement already satisfied: python-dateutil>=2.8.2 in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from pandas->syscourseutils==1.0.6) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from pandas->syscourseutils==1.0.6) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from pandas->syscourseutils==1.0.6) (2024.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from requests->syscourseutils==1.0.6) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from requests->syscourseutils==1.0.6) (3.8)
Requirement already satisfied: urllib3<3,>=1.21.1 in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from requests->syscourseutils==1.0.6) (1.26.20)
Requirement already satisfied: certifi>=2017.4.17 in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (from requests->syscourseutils==1.0.6) (2024.8.30)
Building wheels for collected packages: syscourseutils
  Building wheel for syscourseutils (setup.py) ... done
  Created wheel for syscourseutils: filename=syscourseutils-1.0.6-py3-none-any.whl size=21580 sha256=9efce13a5eae8a7fd0d6a14dc2123117be62ecd35a9d09115ea0daa10a29f200
  Stored in directory: /private/var/folders/8g/px8bm7bj0_j31j71yh6mfd_r0000gn/T/pip-ephem-wheel-cache-6zv5pffq/wheels/74/14/ad/64dd25a774af520d19cd1f28a5e0d167ef58b12046c1fc7572
Successfully built syscourseutils
Installing collected packages: syscourseutils
  Attempting uninstall: syscourseutils
    Found existing installation: syscourseutils 1.0.6
    Uninstalling syscourseutils-1.0.6:
      Successfully uninstalled syscourseutils-1.0.6
Successfully installed syscourseutils-1.0.6

TO confirm it worked, we use its base command

cspt

Usage: cspt [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  badgecounts     check if early bonus is met from output of `gh pr list...
  combinecounts   combine two yaml files by adding values
  createtoyfiles  from a yaml source file create a set of toy files with...
  earlybonus      check if early bonus is met from output of `gh pr list...
  exportac        export ac files for site from lesson
  exporthandout   export prismia version of the content
  exportprismia   export prismia version of the content
  getassignment   get the assignment text formatted
  getbadgedate    cli for calculate badge date
  grade           calculate a grade from yaml that had keys of...
  issuestat
  issuestatus     generate script to appy course issue statuse updates
  kwlcsv          generate the activity file csv file for the site...
  mkchecklist     transform input file to a gh markdown checklist, if the...
  parsedate       process select non dates
  prfixlist       check json output for titles that will not be counted...
  processexport   transform output from mac terminal export to myst...
  progressreport  list PR titles from json or - to use std in that have...
  titlecheck      check a single title

Let’s try the one we saw in the action

cspt getassignment --type prepare

404: Not Found

With default settings, it says 404.

We know that it goes online. So, for example, if you turn your wifi off and then try it again, you will get a different error.

To learn more about this, lets use the --help option.
A lot of CLI tools have this option. In this program, that I made, the Python library click that I used to make this, provides the help option automatically to show the documentation or lets me as the developer add help text to options.

cspt getassignment --help

Usage: cspt getassignment [OPTIONS]

  get the assignment text formatted

Options:
  --type TEXT  type can be {prepare, review, or practice}; default prepare
  --date TEXT  date should be YYYY-MM-DD of the tasks you want; default most
               recently posted
  --help       Show this message and exit.

Now we can see that it can take some options.

If we do not provide a date, I can tell you (and I should add to the documentation), it uses today’s date. Since we ran this on a day with class, before the badges were posted, the file it looked for did not exist, so we got 404.

We can use the options to get the last posted prepare work

cspt getassignment --type prepare --date 2024-09-26

- [ ] Think through and make some notes about what you have learned about design so far. Try to answer the questions below in `design_before.md`.  If you do not now know how to answer any of the questions, write in what questions you have. 

``` 
- What past experiences with making decisions about design of software do you have?
- what experiences studying design do you have? 
- What processes, decisions, and practices come to mind when you think about designing software?
- From your experiences as a user, how you would describe the design of command line tools vs other GUI based tools?
```

gh issue create --help

Create an issue on GitHub.

Adding an issue to projects requires authorization with the `project` scope.
To authorize, run `gh auth refresh -s project`.


USAGE
  gh issue create [flags]

ALIASES
  gh issue new

FLAGS
  -a, --assignee login   Assign people by their login. Use "@me" to self-assign.
  -b, --body string      Supply a body. Will prompt for one otherwise.
  -F, --body-file file   Read body text from file (use "-" to read from standard input)
  -e, --editor           Skip prompts and open the text editor to write the title and body in. The first line is the title and the remaining text is the body.
  -l, --label name       Add labels by name
  -m, --milestone name   Add the issue to a milestone by name
  -p, --project title    Add the issue to projects by title
      --recover string   Recover input from a failed run of create
  -T, --template file    Template file to use as starting body text
  -t, --title string     Supply a title. Will prompt for one otherwise.
  -w, --web              Open the browser to create an issue

INHERITED FLAGS
      --help                     Show help for command
  -R, --repo [HOST/]OWNER/REPO   Select another repository using the [HOST/]OWNER/REPO format

EXAMPLES
  $ gh issue create --title "I found a bug" --body "Nothing works"
  $ gh issue create --label "bug,help wanted"
  $ gh issue create --label bug --label "help wanted"
  $ gh issue create --assignee monalisa,hubot

  $ gh issue create --project "Roadmap"
  $ gh issue create --template "bug_report.md"

LEARN MORE
  Use `gh <command> <subcommand> --help` for more information about a command.
  Read the manual at https://cli.github.com/manual
  Learn about exit codes using `gh help exit-codes`

Interactive Design¶

gh issue create

Creating issue in compsys-progtools/compsys-progtools-fa24-fall24-kwl-template

? Title tst
? Body <Received>
? What's next? Submit
https://github.com/compsys-progtools/compsys-progtools-fa24-fall24-kwl-template/issues/43

gh issue list

Showing 1 of 1 open issue in compsys-progtools/compsys-progtools-fa24-fall24-kwl-template

ID   TITLE  LABELS  UPDATED               
#43  tst            less than a minute ago

gh issue view 43

tst compsys-progtools/compsys-progtools-fa24-fall24-kwl-template#43
Open • brownsarahm opened about 1 minute ago • 0 comments


  No description provided


View this issue on GitHub: https://github.com/compsys-progtools/compsys-progtools-fa24-fall24-kwl-template/issues/43

COmment also allows us to work interactively.

gh issue comment 43

It uses nano which we have already been using, so you are well prepared for this!

- Press Enter to draft your comment in nano... 
? Submit? Yes
https://github.com/compsys-progtools/compsys-progtools-fa24-fall24-kwl-template/issues/43#issuecomment-2377554046

We can look again

gh issue view 43

tst compsys-progtools/compsys-progtools-fa24-fall24-kwl-template#43
Open • brownsarahm opened about 2 minutes ago • 1 comment


  No description provided


brownsarahm • 0m • Newest comment

  lksjflakjfladksjlf;kwhf;kwh                                                 


View this issue on GitHub: https://github.com/compsys-progtools/compsys-progtools-fa24-fall24-kwl-template/issues/43

We can also close issues

gh issue close 43

✓ Closed issue compsys-progtools/compsys-progtools-fa24-fall24-kwl-template#43 (tst)

Now with the pipe¶

cspt getassignment --type prepare --date 2024-09-26 | gh issue create --title 'duplicatee' --body-file -

Creating issue in compsys-progtools/compsys-progtools-fa24-fall24-kwl-template

https://github.com/compsys-progtools/compsys-progtools-fa24-fall24-kwl-template/issues/44

we can aloso open t in browser to see more fully

gh issue view 44 --web

Opening github.com/compsys-progtools/compsys-progtools-fa24-fall24-kwl-template/issues/44 in your browser.

Since your repos are forks, they have two remotes, or upstream repositories, that you can pull from.

We can see with git remote

git remote

origin
upstream

it has a verbose option with the -v flag that shows more detail

git remote -v

origin	https://github.com/compsys-progtools/fall24-brownsarahm.git (fetch)
origin	https://github.com/compsys-progtools/fall24-brownsarahm.git (push)
upstream	https://github.com/compsys-progtools/compsys-progtools-fa24-fall24-kwl-template.git (fetch)
upstream	https://github.com/compsys-progtools/compsys-progtools-fa24-fall24-kwl-template.git (push)

We can change which of those is a default

gh repo set-default origin

expected the "[HOST/]OWNER/REPO" format, got "origin"

I forgot the format and tried to use the short name, origin when gh requires it tobe in owner/repo format, but this error is informative, so we know what to do:

Important

Do this step!! It will fix some of your errors


```{code-cell} bash
:tags: ["skip-execution"]
gh repo set-default compsys-progtools/fall24-brownsarahm

✓ Set compsys-progtools/fall24-brownsarahm as the default repository for the current directory

Hack the course¶

build and explore ideas

Develop new command line tools, github actions, VS Code (or other IDE) extensions, visual aids, etc that help future students in the course. This is a way for you to demonstrate your learning and contribute to open source repos that you can then link to on a portfolio website or resume.

You can participate in this at different levels (or multiple):

build: writing, documenting at the API level, and configuring major pieces (can be collaborative)
explore: writing example/tutorial style docs that another student (or team) build (mostly solo; open to justification for collab)
explore: adding summary/visuals to the notes
community: (test option): testing other students contributions and making descriptive issues or writing short reviews (~3-5 sentenccs); must not be spammy
community: (review option): reviewing PRs submitted by a classmate (can be within a collaborative build, but must be not your own code) a single PR can have multiple reivews if sensible and not duplicative; no spam
community: (contribution) add a glossary term to this site

see the bonus table for more information

Prepare for Next Class¶

review the notes on what is a commit. In gitdef.md on the branch for this issue, try to describe git in the four ways we described a commit. the point here is to think about what you know for git and practice remembering it, not “get the right answer”; this is prepare work, we only check that it is complete, not correct
Start recording notes on how you use IDEs for the next couple of weeks using the template file below. We will come back to these notes in class later, but it is best to record over a time period instead of trying to remember at that time. Store your notes in your fall25 repo in idethoughts.md on a dedicated ide_prep branch. This step is prep for after a few weeks from now, not for Sept 30; keep this branch open until it is specifically asked for

Badges¶

Review

Read today’s notes when they are posted.
Most real projects partly adhere and at least partly deviate from any major design philosophy or paradigm. Review the open source project you looked at for the software.md file from before and decide if it more adheres to or deviates from the unix philosophy. Add a ## Unix Philosophy <Adherance/Deviation> section to your software.md, setting the title to indicate your decision and explain your decision in that section. Provide at least two specific examples, using links to specific lines of code or specific sections in the documentation that support your claims.
create methods.md and answer the following:

- Reflecting on your own programming practice, is initial understanding of concepts or debugging later more challenging for you? and why? Your answer here will be "your chosen context" for the rest of the questions. 
- which of the three methods for studying a system do you use most often in your chosen context? Give an example, like an anectdote of learnign something new or a tricky bug you solved. 
- do you think using a different strategy might help you in your chosen context sometimes? why or why not?

Most real projects partly adhere and at least partly deviate from any major design philosophy or paradigm. Add a ## Unix Philosophy section to your software.md about how that project adheres to and deviates from the unix philosophy. Be specific, using links to specific lines of code or specific sections in the documentation that support your claims. Provide at least one example of both adhering and deviating from the philosophy and three total examples (that is 2 examples for one side and one for the other). You can see what badge software.md was previously assigned in and the original instructions on the KWL file list.
create methods.md and answer the following:

- which of the three methods for studying a system do you use most often when debugging? Give an example
- which do you use when you are trying to understand something new? Is it the same as when you are debugging? why or why not
- do you think the ones you use most often are consistently effective? why or why not? When do they work or not work? (for either learning new thing or debugging)
- what are you most interested in trying that might be different for either learning new things or debugging?




## Questions After Today's Class 

### Is there anything you can do in Github that you can't do locally?

View PRs linked to issues easily is one example.  It's not a lot though


### What should I do for prepare work?

[see instructions](prepare-experience-process)

### What is the `cspt` in get an assignment and what exactly is the `getassignment` command doing?

[see its docs](https://compsys-progtools.github.io/courseutils/cli.html#cspt-getassignment) or [source](https://github.com/compsys-progtools/courseutils/blob/main/cspt/cli.py#L55)

### Why did we type in the codepsace terminal and not on Bash?

the terminal in the codespace is `bash` on linux, where you all have *exactly* the same specs. 

GitBash, the windows terminal program that allows you to use `bash` and `git` commands has
and issue with [mintty]() that you probably have to reinstall it to make `gh` work there, but
the GitHub Codespace comes with `gh` installed. 

### Why did we use the pip install command?

`pip` is the package manager for python. `pip install` installs a package. We installed from the
[source repo]([](https://github.com/compsys-progtools/courseutils))

### What are some examples of dev tools that we can use today 

All the ones we use each class, every IDE you use, etc. 

### When should I use `--help` option?


Whenever you do not know the usage of a command. Not all programs provide it, but many modern ones do. 
Worst case it gives an error.