When do I get an advantage from the terminal?

so far we have used git and bash to accomplish familiar goals, and git and bash feel like just extra work for familiar goals.

We will do this by working in the gh-inclass repo on organizing a set of files. This is still a familiar task, but we will show how doing on the terminal can make it faster^[1], more robust to errors, and automatable^[2]

Setting the stage¶

To prepare for today’s class you examined an open source project.

We noticed that while the contents inside and the distribution of languages used as well as the specific code, or the content of the files was all different, a lot of the organization was similar.

Most had certain community health files and basic info files:

• CONTRIBUTING

• CODE_OF_CONDUCT

• README.MD

• LICENSE

• GOVERNANCE.MD

Setup¶

First, we’ll go back to our github inclass folder

cd gh-inclass-brownsaram

ls

about.md	README.md

and check in with git

git status

On branch fun_fact
Your branch is up to date with 'origin/fun_fact'.

nothing to commit, working tree clean

and then get updated

git pull

1
2
3
4
5
6
7
8
remote: Enumerating objects: 18, done.
remote: Counting objects: 100% (18/18), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 18 (delta 0), reused 18 (delta 0), pack-reused 0 (from 0)
Unpacking objects: 100% (18/18), 1.64 KiB | 152.00 KiB/s, done.
From https://github.com/compsys-progtools/gh-inclass-fa25-brownsarahm
 * [new branch]      organizing_ac -> origin/organizing_ac
Already up to date.

Recall, we can use git branch with no inputs to see what branches we have locally

git branch

  1-add-a-readme
  fun_fact
* main
  my_branch

the new branch does not show, becuase while it has been pulled and git can tell what it should be, we have not used it locally so it does not really exist yet locally.

To see branches that are in the remote we use the -r flag.

git branch -r

1
2
3
4
5
  origin/1-add-a-readme
  origin/HEAD -> origin/main
  origin/fun_fact
  origin/main
  origin/organizing_ac

Now we have the new branch!

We can switch to it with checkout

git checkout organizing_ac

branch 'organizing_ac' set up to track 'origin/organizing_ac'.
Switched to a new branch 'organizing_ac'

Now, if we look at the contents of the working direcotry again, to see what the new branch includes.

ls

_config.yml		LICENSE.md
_toc.yml		philosophy.md
abstract_base_class.py	README.md
alternative_classes.py	scratch.ipynb
API.md			setup.py
CONTRIBUTING.md		tests_alt.py
example.md		tests_helpers.py
helper_functions.py	tests_imp.py
important_classes.py	tsets_abc.py

Organizing a project (working with files)¶

A common question is about how to organize projects. While our main focus in this class session is the bash commands to do it, the task that we are going to do is to organize a hypothetical python project

Put another way, we are using organizing a project as the context to motivate practicing with bash commands for moving files.

A different the instructor might go through a slide deck that lists commands and describes what each one does and then have examples at the end. Instead, we are going to focus on organizing files, and I will introduce the commands we need along the ways.

next we are going to pretend we worked on the project and made a bunch of files

I gave a bunch of files, each with a short phrase in them.

• none of these are functional files

• the phrases mean you can inspect them on the terminal

these are all actually plain text files

cat concatenates the contents of a file to STDOUT, which can be thought of like a special file that our terminal reads

so for example:

cat API.md

jupyterbook file to generate api documentation

or:

cat scratch.ipynb

jupyter notebook from dev

we will create a new branch before we start doing some new work. We do not have to have this branch off of main, it can be off of the one we are already on. Remember branches are not copies they are pointers to different specific commits.

git  checkout -b organization

Switched to a new branch 'organization'

and confirm it is as we expect.

git status

On branch organization
nothing to commit, working tree clean

Manipulating the Streams¶

The STDOUT stream can be manipulated. Any time we open or manipulate a file, it goes through a stream. So we can use other files in place of how we by default use STDOUT.

To try this out, let’s first recall what is in the README file.

cat README.md

# GitHub practice

test

echo is a simple command that repeats what you have done, again by default to STDOUT.

echo "hello"

hello

not so exciting, but there is a way we can use this!

We can also redirect (send) the contents of a command from stdout to a file in bash. Like file operations while programming there is a similar concept to this mode.

There are two types of redirects, like there are two ways to write to a file, more generally:

• overwrite (>)

• append (>>)

These sybols are each called a redirect

Redirect in append mode¶

We can use a to send output of any command to a different stream than STDOUT. So for example, we can send a phrase into the README.md file.

echo "today is rainy" >> README.md

This time there was no output, since nothing when to STDOUT.

Then we check the contents of the file and we see that the new content is there.

cat README.md

1
2
3
4
# GitHub practice

test
today is rainy

the new content!

Redirect in write mode¶

We can redirect other commands too. This time we will use one > to use write mode instead of append mode.

git status > curgit

Again no output

The curgit^[3] file did not exist before, so let’s see if it is there.

ls

_config.yml		LICENSE.md
_toc.yml		philosophy.md
abstract_base_class.py	README.md
alternative_classes.py	scratch.ipynb
API.md			setup.py
CONTRIBUTING.md		tests_alt.py
curgit			tests_helpers.py
example.md		tests_imp.py
helper_functions.py	tsets_abc.py
important_classes.py

it’s there

and we can look at its contents too

cat curgit

On branch organization
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   README.md

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	curgit

no changes added to commit (use "git add" and/or "git commit -a")

Connected bash commands work right to left¶

From the contents of the file, we can see something really important about how the redirect (and other ways of chaining bash commands together) work.

Notice that above, the curgit file is mentioned in the git status that is in the contents of the curgit file. We didn’t create the file on purpose before we ran the line git status > curgit. This is because bash works partially right to left and then resolves after. So, what happens is:

1. the file curgit is created and a stream to it is opened

2. the STDOUT stream is connected to that stream

3. git status runs, it compares the working directory to the last commit, sends its output to STDOUT which is connected to curgit

4. the file stream is closed.

this is not a file we actually want, which gives us a chance to learn another new bash command: rm for remove, removes the entry for this file from the file table. Your computer keeps track of all of the paths and where in the disk (like the memory address) the corresponding content is. if we have no entry, we cannot, using normal tools, find the contents, though it is not erased from the disk or overwritten. The OS is then free to overwrite those bits if needed.

rm curgit

Let’s check in with git

git status

On branch organization
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   README.md

no changes added to commit (use "git add" and/or "git commit -a")

Now we have made some changes we want, so let’s commit our changes.

git commit -a -m 'add note to readme'

[organization 285dd21] add note to readme
 1 file changed, 1 insertion(+)

and confirm a clean working directory

git status

On branch organization
nothing to commit, working tree clean

Git can save you from mistakes¶

Let’s try another redirect. With two we append to the file, with one, in write mode for a file with pre-exisiting content, let’s see:

echo "rain is sad" > README.md

andd then check the file

cat README.md

rain is sad

It wrote over the old content with this new content. This would be bad, we lost content, but this is what git is for!

It is very very easy to undo work since our last commit.

This is good for times when you have something you have an idea and you do not know if it is going to work, so you make a commit before you try it. Then you can try it out. If it doesn’t work you can undo and go back to the place where you made the commit.

To do this, we will first check in with git

git status

1
2
3
4
5
6
7
On branch organization
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   README.md

no changes added to commit (use "git add" and/or "git commit -a")

Notice that it tells us what to do (use "git restore <file>..." to discard changes in working directory). The version of README.md that we broke is in the working directory but not commited to git, so git refers to them as “changes” in the working directory.

We run that command

git restore README.md

this command has no output, so we can use git status to check first

and we can check the file

cat README.md

# GitHub practice

test
today is rainy

Back how we wanted it!

Manipulating files with bash¶

Next we will add some descriptive content to the README

 echo "|file | contents |
> > | --| -- |
> > | abstract_base_class.py | core abstract classes for the project |
> > | helper_functions.py | utitly funtions that are called by many classes |
> > | important_classes.py | classes that inherit from the abc |
> > | alternative_classes.py | classes that inherit from the abc |
> > | LICENSE.md | the info on how the code can be reused|
> > | CONTRIBUTING.md | instructions for how people can contribute to the project|
> > | setup.py | file with function with instructions for pip |
> > | test_abc.py | tests for constructors and methods in abstract_base_class.py|
> > | tests_helpers.py | tests for constructors and methods in helper_functions.py|
> > | tests_imp.py | tests for constructors and methods in important_classes.py|
> > | tests_alt.py | tests for constructors and methods in alternative_classes.py|
> > | API.md | jupyterbook file to generate api documentation |
> > | _config.yml | jupyterbook config for documentation |
> > | _toc.yml | jupyter book toc file for documentation |
> > | philosophy.md | overview of how the code is organized for docs |
> > | example.md | myst notebook example of using the code |
> > | scratch.ipynb | jupyter notebook from dev |" >> README.md

this explains each file a little bit more than the name of it does. We see there are sort of 5 groups of files:

• about the project/repository

• code that defines a python module

• test code

• documentation

• extra files that “we know” we can delete.

We also learn something about bash: using the open quote " then you stay inside that until you close it. when you press enter the command does not run until after you close the quotes

let’s look at the file again

cat README.md

# GitHub practice

test
today is rainy
|file | contents |
> | --| -- |
> | abstract_base_class.py | core abstract classes for the project |
> | helper_functions.py | utitly funtions that are called by many classes |
> | important_classes.py | classes that inherit from the abc |
> | alternative_classes.py | classes that inherit from the abc |
> | LICENSE.md | the info on how the code can be reused|
> | CONTRIBUTING.md | instructions for how people can contribute to the project|
> | setup.py | file with function with instructions for pip |
> | test_abc.py | tests for constructors and methods in abstract_base_class.py|
> | tests_helpers.py | tests for constructors and methods in helper_functions.py|
> | tests_imp.py | tests for constructors and methods in important_classes.py|
> | tests_alt.py | tests for constructors and methods in alternative_classes.py|
> | API.md | jupyterbook file to generate api documentation |
> | _config.yml | jupyterbook config for documentation |
> | _toc.yml | jupyter book toc file for documentation |
> | philosophy.md | overview of how the code is organized for docs |
> | example.md | myst notebook example of using the code |
> | scratch.ipynb | jupyter notebook from dev |

First, we’ll make a directory with mkdir

mkdir docs

next we will move a file there with mv

mv philosophy.md  docs/

what this does is change the path of the file from gh-inclass-fa25-brownsarahm/philosophy.md to gh-inclass-fa25-brownsarahm/docs/philosophy.md

It does not rewrite the file contents to disk, so even for large files this is quick, it changes only the file table that stores each path and a disk address.

we can look in the docs folder

cd docs/

and use ls

ls

philosophy.md

or go back

cd ..

and use ls with the relative path to where we moved it

ls docs/

philosophy.md

we can also check the original location

ls

_config.yml		important_classes.py
_toc.yml		LICENSE.md
abstract_base_class.py	README.md
alternative_classes.py	scratch.ipynb
API.md			setup.py
CONTRIBUTING.md		tests_alt.py
docs			tests_helpers.py
example.md		tests_imp.py
helper_functions.py	tsets_abc.py

not there!

Moving multiple files with patterns¶

We can use the * wildcard operator to move all files that match the pattern. We’ll start with the two yml (yaml) files that are both for the documentation.

mv *.yml docs/

Again, we confirm it worked by seeing that they are no longer in the working directory.

ls

abstract_base_class.py	LICENSE.md
alternative_classes.py	README.md
API.md			scratch.ipynb
CONTRIBUTING.md		setup.py
docs			tests_alt.py
example.md		tests_helpers.py
helper_functions.py	tests_imp.py
important_classes.py	tsets_abc.py

and that they are in docs

ls docs/

_config.yml	_toc.yml	philosophy.md

We see that most of the test files start with tests_ but one starts with tsets_. We can fix this!

Renaming files¶

We can use mv to change the name as well. This is because “moving” a file and is really about changing its path, not actually copying it from one location to another and the file name is a part of the path.

mv tsets_abc.py tests_abc.py

This changes the path from .../tsets_abc.py to .../tests_abc.py to. It is doing the same thing as when we use it to move a file from one folder to another folder, but changing a different part of the path.

we can use ls to check that it worked

ls

abstract_base_class.py	LICENSE.md
alternative_classes.py	README.md
API.md			scratch.ipynb
CONTRIBUTING.md		setup.py
docs			tests_abc.py
example.md		tests_alt.py
helper_functions.py	tests_helpers.py
important_classes.py	tests_imp.py

Now we make a new folder:

mkdir tests

and move all of the test files there:

mv tests_* tests/

Hidden files¶

We saw before that some files and folders are hidden

and that to see them with ls we need the -a flag

ls -a

.			example.md
..			helper_functions.py
.git			important_classes.py
.github			LICENSE.md
abstract_base_class.py	README.md
alternative_classes.py	scratch.ipynb
API.md			setup.py
CONTRIBUTING.md		tests
docs

We are going to make a special hidden file and an extra one. We will use the following command:

touch .secret .gitignore

We also learn 2 things about touch and bash:

• touch can make multiple files at a time

• lists in bash are separated by spaces and do not require brackets

lets put some content in the secret file, this is simple text, but it represents a real type of content, things we need but do not want to share with collaborators or make public.

echo "my dev secret" > .secret

again, we still do not see the file, it’s hidden from a casual user already!

ls

abstract_base_class.py	important_classes.py
alternative_classes.py	LICENSE.md
API.md			README.md
CONTRIBUTING.md		scratch.ipynb
docs			setup.py
example.md		tests
helper_functions.py

but we can see them both

ls -a

.			docs
..			example.md
.git			helper_functions.py
.github			important_classes.py
.gitignore		LICENSE.md
.secret			README.md
abstract_base_class.py	scratch.ipynb
alternative_classes.py	setup.py
API.md			tests
CONTRIBUTING.md

and the file is normally visible to all bash commands

cat .secret

my dev secret

Let’s check in with git

git status

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
On branch organization
Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   README.md
	deleted:    _config.yml
	deleted:    _toc.yml
	deleted:    philosophy.md
	deleted:    tests_alt.py
	deleted:    tests_helpers.py
	deleted:    tests_imp.py
	deleted:    tsets_abc.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	.gitignore
	.secret
	docs/
	tests/

no changes added to commit (use "git add" and/or "git commit -a")

git sees the file but it is not yet tracked. If we were to use git add . it would start getting tracked. We do not want that and we do not want to have to manually add each other file and just avoid this one.

gitignore lets us not track certain files

let’s ignore that .secret file

echo ".secret" >>.gitignore

Now we check with git again:

git status

On branch organization
Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   README.md
	deleted:    _config.yml
	deleted:    _toc.yml
	deleted:    philosophy.md
	deleted:    tests_alt.py
	deleted:    tests_helpers.py
	deleted:    tests_imp.py
	deleted:    tsets_abc.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	.gitignore
	docs/
	tests/

no changes added to commit (use "git add" and/or "git commit -a")

no more secret!

the file is still there:

cat .secret

my dev secret

Tracking moved files¶

Now we want to add the rest of the files we did, notice that above in the git status it thought we had deleted a lot of files that we had actually moved.

Let’s add one of the folders for tracking

git add docs/

git status

On branch organization
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
	new file:   docs/_config.yml
	new file:   docs/_toc.yml
	new file:   docs/philosophy.md

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   README.md
	deleted:    _config.yml
	deleted:    _toc.yml
	deleted:    philosophy.md
	deleted:    tests_alt.py
	deleted:    tests_helpers.py
	deleted:    tests_imp.py
	deleted:    tsets_abc.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	.gitignore
	tests/

it still thinks the files are deleted, but also that there are new files.

Now let’s add all

git add .

and status

git status

On branch organization
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
	new file:   .gitignore
	modified:   README.md
	renamed:    _config.yml -> docs/_config.yml
	renamed:    _toc.yml -> docs/_toc.yml
	renamed:    philosophy.md -> docs/philosophy.md
	renamed:    tsets_abc.py -> tests/tests_abc.py
	renamed:    tests_alt.py -> tests/tests_alt.py
	renamed:    tests_helpers.py -> tests/tests_helpers.py
	renamed:    tests_imp.py -> tests/tests_imp.py

Now, it knows that we have actually renamed the files.

We had to tell it to track the changes to both . and docs for it to detect that they were the same files that we had only moved. It compares files using their names but also the contents. We will pick up from this idea more next week, but this is an important observation.

Finally, we will commit

git commit -m 'begin reorg
> '

[organization e899a0e] begin reorg
 9 files changed, 20 insertions(+)
 create mode 100644 .gitignore
 rename _config.yml => docs/_config.yml (100%)
 rename _toc.yml => docs/_toc.yml (100%)
 rename philosophy.md => docs/philosophy.md (100%)
 rename tsets_abc.py => tests/tests_abc.py (100%)
 rename tests_alt.py => tests/tests_alt.py (100%)
 rename tests_helpers.py => tests/tests_helpers.py (100%)
 rename tests_imp.py => tests/tests_imp.py (100%)

Recap¶

Why do I need a terminal

1. replication/automation

2. it’s always there and doesn’t change

3. it’s faster one you know it (also see above)

Prepare for Next Class¶

1. Bring git questions or scenarios you want to be able to solve to class (in your mind or comment here if that helps you remember)

2. Try read and understand the workflow files in your KWL repo, the goal is not to be sure you understand every step, but to get an idea about the big picture ideas and just enough to complete the following. Try to modify files, on a prepare branch, so that your name is already filled in and AymanBx or thomaspeck11 (whoever you sit closer to in class) is already requested as a reviewer when your experience badge (inclass) action runs. We will give the answer in class, but especially do not do this step on the main branch it could break your action.

Hints:

• Look for bash commands that we have seen before

• cp copies a file

• check what the experience makeup action does

Badges¶

# Terminal File moving reflection
1. How was this activity overall Did this get easier toward the end?
2. How was it different working on your own computer compared to the Codespace form? 
3. Did you have to look up how to do anything we had not done in class?
4. When do you think that using the terminal will be better than using your GUI file explorer?
5. What questions/challenges/ reflections do you have after this?
6. Append all of the commands you used in lab below. (not from your local computer's history, from the codespace history)

Experience Report Evidence¶

Save your history with:

history > activity-2025-09-18.md

then append your git status, and the contents of your github-in-class and github-in-class/docs with to help visually separate the parts.

echo "***--" >> activity-2025-09-18.md
git status >> activity-2025-09-18.md
echo "***--" >> activity-2025-09-18.md
ls >> activity-2025-09-18.md
echo "***--" >> activity-2025-09-18.md
ls docs/ >> activity-2025-09-18.md

then edit that file (on terminal, any text editor, or an IDE) to make sure it only includes things from this activity.

Questions After Today’s Class¶

How much longer is the grace period for badges?¶

from cspt import EARLY_BIRD_DEADLINE_STR

The penalty free zone ends on 2025-09-25

I’m interested in learning more about wildcard operators.¶

explore They are an example of a type of o pattern matching. The most general is regular expressions. This is a good explore badge topic.

Do patterns and the wild card operator (*) apply to all terminal commands?¶

I am sure some exist that do not, especially since people can make their own, but most bash commands that take a file as input do accept this.