446 lines
16 KiB
Markdown
446 lines
16 KiB
Markdown
# Chapter 5. Saving User Input: Testing the Database
|
|
|
|
Benefit of TDD: iterative style of development.
|
|
|
|
Our target: take to-do item input from user and sent it to server
|
|
|
|
## 5.1 Wiring Up Our Form to Send a POST Request
|
|
|
|
To let browser send a standard HTML POST request, steps needed:
|
|
1. Give `<input>` element a `name=` attribute
|
|
2. Wrap it in a `<form>` tag with `method="POST"`
|
|
|
|
several tricks used to debug unexpected failure in functional test:
|
|
* Add print statements, to show, for example, what the current page text is.
|
|
* Improve the error message to show more info about the current state.
|
|
* Manually visit the site yourself.
|
|
* Use `time.sleep` to pause the test during execution (Chosen to be used)
|
|
|
|
|
|
|
|
* Reason of failure: CSRF token missed.
|
|
* What's CSRF token:
|
|
* > a unique, secret, unpredictable value that is generated by the server-side application and transmitted to the client in such a way that it is included in a subsequent HTTP request made by the client.
|
|
* One of Django's CSRF protection methods: placing a little auto-generated token into each generated form, so the form can be identified as POST request coming from original site.
|
|
|
|
Embed CSRF token using `{% csrf_token %}
|
|
|
|
```html
|
|
<form method="POST">
|
|
<input name="item_text" id="id_new_item" placeholder="Enter a to-do item" />
|
|
{% csrf_token %}
|
|
</form>
|
|
```
|
|
|
|
Result of running FT, page is layed with content but failed test, as server haven't wired to deal with POST request yet.
|
|
|
|
## 5.2 Processing a POST Request on the Server
|
|
|
|
Current `home.html` don't have `action=` attribute in `<form>`. Hence, it is submitting back to the `/` page (the same URL it was rendered from). It means the request is dealt by `home_page` function.
|
|
|
|
Add a test case into `lists/tests.py`
|
|
|
|
```python
|
|
def test_can_save_a_POST_request(self):
|
|
response = self.client.post('/', data={'item_text': 'A new list item'})
|
|
self.assertIn('A new list item', response.content.decode())
|
|
```
|
|
|
|
Test it will return error
|
|
|
|
```
|
|
AssertionError: 'A new list item' not found in '<html>\n <head>\n <title>To-Do lists</title>\n </head>\n <body>\n <h1>Your To-Do list</h1>\n <form method="POST">\n <input name="item_text" id="id_new_item" placeholder="Enter a to-do item" />\n <input type=\'hidden\' name=\'csrfmiddlewaretoken\' value=\'U3BKUu5Xl50FQMi8JNz5ZCCIkHKISJJ2QtVuLuZ3zdRmOWXh7hvwZfXIh76CQiN2\' />\n </form>\n\n <table id="id_list_table"></table>\n </body>\n</html>'
|
|
```
|
|
|
|
We will pass variables from Python view code into HTML templates
|
|
|
|
## 5.3 Passing Python Variables to Be Rendered in the Template
|
|
|
|
* Django's template syntax lets us include a python object in template using notation `{{ ... }}`
|
|
* renderer will display the object as a string
|
|
|
|
```html
|
|
<body>
|
|
<h1>Your To-Do list</h1>
|
|
<form method="POST">
|
|
<input name="item_text" id="id_new_item" placeholder="Enter a to-do item" />
|
|
{% csrf_token %}
|
|
</form>
|
|
|
|
<table id="id_list_table">
|
|
<tr><td>{{ new_item_text }}</td></tr>
|
|
</table>
|
|
</body>
|
|
```
|
|
* `new_item_text` is the variable name for the user input
|
|
|
|
Modify test case in `lists/tests.py`, `assertTemplateUsed` will check whether we are still using template
|
|
|
|
```python
|
|
def test_can_save_a_POST_request(self):
|
|
response = self.client.post('/', data={'item_text': 'A new list item'})
|
|
self.assertIn('A new list item', response.content.decode())
|
|
self.assertTemplateUsed(response, 'home.html')
|
|
```
|
|
|
|
return error
|
|
|
|
```
|
|
self.assertTemplateUsed(response, 'home.html')
|
|
File "/home/jason/miniconda3/envs/python-tdd-book/lib/python3.6/site-packages/django/test/testcases.py", line 578, in assertTemplateUsed
|
|
self.fail(msg_prefix + "No templates used to render the response")
|
|
AssertionError: No templates used to render the response
|
|
```
|
|
|
|
QUES: Why fail?
|
|
|
|
ANS: In `lists/views.py`, `home_page` function no longer return rendered page. When facing `POST` request, it return a HttpResponse
|
|
|
|
```python
|
|
if request.method == 'POST':
|
|
return HttpResponse(request.POST['item_text'])
|
|
return render(request, 'home.html')
|
|
```
|
|
|
|
QUES: How to fix it?
|
|
|
|
ANS: Modify `home_page` method to return rendered page.
|
|
|
|
QUES: How to use `render` function?
|
|
|
|
ANS: `render` function takes, (3rd arguments), a dictionary which maps template variable names to their values.
|
|
|
|
```python
|
|
def home_page(request):
|
|
return render(request=request, template_name='home.html', context={
|
|
'new_item_text': request.POST['item_text'],
|
|
})
|
|
```
|
|
|
|
### 5.3.1 An Unexpected Failure
|
|
|
|
Running test as shown above will return error
|
|
|
|
```
|
|
ERROR: test_uses_home_template (lists.tests.HomePageTest)
|
|
...
|
|
File "/home/jason/miniconda3/envs/python-tdd-book/lib/python3.6/site-packages/django/utils/datastructures.py", line 85, in __getitem__
|
|
raise MultiValueDictKeyError(repr(key))
|
|
django.utils.datastructures.MultiValueDictKeyError: "'item_text'"
|
|
```
|
|
|
|
Error is from `test_uses_home_template` test case. We broke the code path where there is no POST request; i.e. explain below
|
|
* `test_can_save_a_POST_request` get POST request with `'/', data={'item_text': 'A new list item'}`
|
|
* `test_uses_home_template` get POST request with only `'/'`, while `request.POST['item_text']` cannot find it.
|
|
|
|
QUES: How to fix it?
|
|
|
|
ANS: type of `request.POST` is 'django.http.request.QueryDict' is a dictionary. So dictionary method `dict.get` can be used to supply a default value `''`
|
|
|
|
```python
|
|
def home_page(request):
|
|
return render(request=request, template_name='home.html', context={
|
|
'new_item_text': request.POST.get('item_text', ''),
|
|
})
|
|
```
|
|
|
|
...
|
|
|
|
## 5.4 Three Strikes and Refactor
|
|
|
|
Principle: **Don't Repeat Yourself (DRY)**, or *three strikes and refactor (事不过三)*; 同一个代码段重复三次以后就应该 remove duplication
|
|
|
|
* QUES: What we do?
|
|
* ANS: 将重复的 `table = ... rows = ... self.assertIn ...` 独立为 helper function (as shown below)
|
|
|
|
```python
|
|
def check_for_row_in_list_table(self, row_text):
|
|
table = self.browser.find_element_by_id('id_list_table')
|
|
rows = table.find_elements_by_tag_name('tr')
|
|
self.assertIn(row_text, [row.text for row in rows])
|
|
```
|
|
|
|
## 5.5 The Django ORM and Our First Model
|
|
|
|
* **Object-Relational Mapper (ORM)** is a layer of abstraction of data stored in database (tables, rows, and columns)
|
|
* ORM let use work with database using OOP.
|
|
* *Classes map to database tables*
|
|
* *Attributes map to columns*
|
|
* *Individual instance of the class is mapped to a row of data in database*
|
|
|
|
Django comes with ORM. Using which, creating a new record in data is easy:
|
|
1. creating an object
|
|
2. assigning some attributes
|
|
3. calling `.save()` function.
|
|
|
|
Django also provide API for:
|
|
* querying the database using class attributes of object `.object`
|
|
* use simplest possible query `.all()` to retrieves all records for the table
|
|
* returned object is a list-like object called 'QuerySet'
|
|
* Using 'QuerySet', we can extract individual objects
|
|
|
|
All these can be used in test case
|
|
|
|
```python
|
|
class ItemModelTest(TestCase):
|
|
|
|
def test_saving_and_retrieving_items(self):
|
|
first_item = Item()
|
|
first_item.text = 'The first (ever) list item'
|
|
first_item.save()
|
|
|
|
second_item = Item()
|
|
second_item.text = 'Item the second'
|
|
second_item.save()
|
|
|
|
saved_items = Item.objects.all()
|
|
self.assertEqual(saved_items.count(), 2)
|
|
|
|
first_saved_item = saved_items[0]
|
|
second_saved_item = saved_items[1]
|
|
self.assertEqual(first_saved_item.text, 'The first (ever) list item')
|
|
self.assertEqual(second_saved_item.text, 'Item the second')
|
|
```
|
|
|
|
### 5.5.1 Our First Database Migration
|
|
|
|
* In Django, ORM's job is to model the database,
|
|
* **migration** is a system that build the database. It has ability to add/remove tables and columns, based changes on `models.py` file. Like a version control system for the database.
|
|
|
|
Steps to update database:
|
|
1. `python manage.py makemigrations` to create `migrations/000x_initials.py`
|
|
2. Make migration via `python manage.py migrate`
|
|
|
|
### 5.5.2 The Test Gets Surprisingly Far
|
|
|
|
To pass the test, add class `Item`:
|
|
|
|
```python
|
|
class Item(models.Model):
|
|
text = models.TextField(default='')
|
|
```
|
|
|
|
Then migrate the database again, and then we can pass the test
|
|
|
|
## 5.6 Saving the POST to the Database
|
|
|
|
Modify the POST request test case that view will save a new item to the database instead of just passing to response.
|
|
|
|
```python
|
|
def test_can_save_a_POST_request(self):
|
|
response = self.client.post('/', data={'item_text': 'A new list item'})
|
|
|
|
self.assertEqual(Item.objects.count(), 1) # check that one new Item has been saved to the database
|
|
new_item = Item.objects.first() # same as doing objects.all()[0]
|
|
self.assertEqual(new_item.text, 'A new list item') # check that the item's text is correct
|
|
|
|
self.assertIn('A new list item', response.content.decode())
|
|
self.assertTemplateUsed(response, 'home.html')
|
|
```
|
|
|
|
First step to pass the test: modify `home_page` function to save request (including empty request just to '/')
|
|
|
|
```python
|
|
def home_page(request):
|
|
item = Item()
|
|
item.text = request.POST.get('item_text', '')
|
|
item.save()
|
|
|
|
return render(request=request, template_name='home.html', context={
|
|
'new_item_text': request.POST.get('item_text', ''),
|
|
})
|
|
```
|
|
|
|
To-Do List to modify code (sometimes make sure code pass is ok, modify later):
|
|
* Don't save blank items for every request (empty request)
|
|
* Code small: POST test is too long?
|
|
* Display multiple items in the table
|
|
* Support more than one list
|
|
|
|
Solve the first todo list by creating a UT:
|
|
|
|
```python
|
|
def test_only_saves_items_when_necessary(self):
|
|
self.client.get('/')
|
|
self.assertEqual(Item.objects.count(), 0)
|
|
```
|
|
|
|
Running test will result `AssertionError: 1 != 0`
|
|
|
|
We can pass this UT via modifying `home_page`:
|
|
|
|
```python
|
|
def home_page(request):
|
|
if request.method == 'POST':
|
|
new_item_text = request.POST['item_text'] # use new_item_text to either hold POST contents or empty string
|
|
Item.objects.create(text=new_item_text) # .object.create is shortcut for creating a new Item, without needing to call .save()
|
|
else:
|
|
new_item_text = ''
|
|
|
|
return render(request=request, template_name='home.html', context={
|
|
'new_item_text': request.POST.get('item_text', ''),
|
|
})
|
|
```
|
|
|
|
## 5.7 Redirect After a POST
|
|
|
|
A view func has 2 jobs:
|
|
1. Processing user input (save to database)
|
|
2. Returning an appropriate response
|
|
|
|
2nd one is our target in this section
|
|
|
|
"Always redirect after a POST", change test case to save a POST request. So redirect back to home page, instead of rendering a response with item in it
|
|
|
|
```python
|
|
def test_can_save_a_POST_request(self):
|
|
response = self.client.post('/', data={'item_text': 'A new list item'})
|
|
|
|
self.assertEqual(Item.objects.count(), 1)
|
|
new_item = Item.objects.first()
|
|
self.assertEqual(new_item.text, 'A new list item')
|
|
|
|
self.assertEqual(response.status_code, 302)
|
|
self.assertEqual(response['location'], '/')
|
|
```
|
|
|
|
* QUES: What mean redirect?
|
|
* ANS: After sending POST, the returned response does not contain `.content` that rendered by a template, instead it's HTTP redirect that point to another URL
|
|
|
|
Modify `views.py` to pass this UT
|
|
|
|
```python
|
|
def home_page(request):
|
|
if request.method == 'POST':
|
|
Item.objects.create(text=request.POST['item_text']) # Processing user input
|
|
return redirect('/') # Redirect back to home.html
|
|
|
|
return render(request=request, template_name='home.html')
|
|
```
|
|
|
|
### 5.7.1 Better Unit Testing Practice: Each Test Should Test One Thing
|
|
|
|
Note: good UT practice is that each test should only test one thing.
|
|
|
|
So, separate the test code into two:
|
|
|
|
```python
|
|
def test_can_save_a_POST_request(self):
|
|
self.client.post('/', data={'item_text': 'A new list item'})
|
|
|
|
self.assertEqual(Item.objects.count(), 1)
|
|
new_item = Item.objects.first()
|
|
self.assertEqual(new_item.text, 'A new list item')
|
|
|
|
def test_redirects_after_POST(self):
|
|
response = self.client.post('/', data={'item_text': 'A new list item'})
|
|
|
|
self.assertEqual(response.status_code, 302)
|
|
self.assertEqual(response['location'], '/')
|
|
```
|
|
|
|
## 5.8 Rendering Items in the Template
|
|
|
|
Now, both "Don't save blank items for every request" and "Code smell: POST test is too long?" are finished
|
|
|
|
Target: "Display multiple item in the table"
|
|
|
|
TDD step 1: create a UT to check
|
|
|
|
```python
|
|
|
|
def test_displays_all_list_items(self):
|
|
Item.objects.create(text="itemey 1")
|
|
Item.objects.create(text="itemey 2")
|
|
|
|
response = self.client.get('/')
|
|
|
|
self.assertIn('itemey 1', response.content.decode())
|
|
self.assertIn('itemey 2', response.content.decode())
|
|
```
|
|
|
|
Test will fail as `home.html` we still only allow one item in table
|
|
|
|
```html
|
|
<table id="id_list_table">
|
|
<tr><td>1: {{ new_item_text }}</td></tr>
|
|
</table>
|
|
```
|
|
|
|
Fix it by allow it to have iterating list. Django template syntax has a tag for iterating through a list `{% for .. in .. %}` + `% endfor %`
|
|
|
|
```html
|
|
<table id="id_list_table">
|
|
{% for item in items %}
|
|
<tr><td>1: {{ item.text }}</td></tr>
|
|
{% endfor %}
|
|
</table>
|
|
```
|
|
|
|
Also need pass items to template from home page view:
|
|
|
|
```python
|
|
def home_page(request):
|
|
if request.method == 'POST':
|
|
Item.objects.create(text=request.POST['item_text'])
|
|
return redirect('/')
|
|
|
|
items = Item.objects.all() # get objects from database (model)
|
|
return render(request=request,
|
|
template_name='home.html',
|
|
context={'items': items}) # pass items into template using render
|
|
```
|
|
|
|
## 5.9 Creating Our Production Database with migrate
|
|
|
|
If use FT `functional_tests.py` to verify, will return an error `no such table: lists_item`
|
|
|
|
Why? Because Django creates test database for unit tests, this database cannot be used by FT.
|
|
|
|
Migrate the database using `python manage.py migrate`
|
|
|
|
`functional_tests.py` Test Result:
|
|
|
|
```
|
|
AssertionError: '2: Use peacock feathers to make a fly' not found in ['1: Buy peacock feathers', '1: Use peacock feathers to make a fly']
|
|
```
|
|
|
|
* Problem: need to get list numbering right.
|
|
* Solution: using Django template tag, `forloop.counter` in template `home.html`
|
|
|
|
```html
|
|
<table id="id_list_table">
|
|
{% for item in items %}
|
|
<tr><td>{{ forloop.counter }}: {{ item.text }}</td></tr>
|
|
{% endfor %}
|
|
</table>
|
|
```
|
|
|
|
* Remaining problem: database are polluted by items from previous FT round.
|
|
* ANS: Find a automated way of clear database after each FT
|
|
|
|
```
|
|
rm db.sqlite3
|
|
python manage.py migrate --noinput
|
|
```
|
|
|
|
## Recap
|
|
|
|
Learning outcome:
|
|
* Setup a form to add new items to the list using POST.
|
|
* Setup a simple model in the database to save list items.
|
|
* Creating database migrations, for both test database and real database
|
|
* 2 Django template tags `{% csrf_token %}` & `{$ for ... endfor $}` loop
|
|
|
|
Useful TDD Concepts:
|
|
* **Regression**: When new code breaks some aspect of application (which used to work).
|
|
* **Unexpected failure**: When a test fails unexpectedly. Reason:
|
|
* 1. mistake in our tests;
|
|
* 2. Tests have helped us find a regression, and we need to fix sth in our code.
|
|
* **Red/Green/Refactor**: TDD process in another language.
|
|
* Write a test and see it fail (Red)
|
|
* Write some code to get it pass (Green)
|
|
* Refactor to improve implementation
|
|
* **Triangulation**: Generalizing the implementation by adding new test case to cover new specific e.g.
|
|
* **Three strikes and refactor**: Rule to remove duplication from code.
|
|
* **Scratchpad to-do list**: to-do list to write during coding, and check what we're doing. |