TPS REST interface design
Hi, This is the initial draft of the TPS REST interface design: http://pki.fedoraproject.org/wiki/TPS_REST_Interface Please take a look and let me know if you have any comments. Thanks! -- Endi S. Dewata
1. One of the decisions you made was to allow admins, agents and operators to access the same resources. So, unlike the rest of the java subsystems, we will have /tps/rest/tokens as opposed to /tps/rest/agent/tokens or /tps/rest/admin/tokens One reason one might want to add /agent or /admin is if the content of GET /tps/tokens/X is different for admins/agents/operators. So what is the difference between op=search and op=search_admin, op=view and op=view_admin, op=show and op=show_admin? etc. Is there any case where those differences need be preserved? 2. You use PUT /tps/rest/tokens/X as the operation to modify tokens. Usually REST semantics imply that the resource passed in to the PUT operation essentially replace the resource at the server. Is that your intention? I'm guessing that the resource that would be passed in is the token object returned in GET /tps/rest/tokens/foo. It would help if you defined what a token resource was. Or is your intention to send in something with an "action" parameter - in which case, the correct operation is a POST? 3. Should we be worried about a XSS attack here for modifying the state of the token? My guess is that we need to take advantage of the nonce mechanism, in which case, token state modification will require two steps. 4. You should also note that we will be sending back BAD_REQUEST (401?) when the token state transition is not permitted. 5. In the response to PUT /tokens/X, it is not necessary to return the state of the token. Rather, you should return a URL pointer to the newly modified resource. The same comment applies to the other PUT operations as well. 6. Same question about XSS for add/remove token/ add/remove user. 7. Note that the difference between op=view_activity, op=view_activity_all etc. is in the types of activities that are visible. This should be handled seamlessly in the logic used to select activity records from the db. 8. For the configuration items - audit config, profiles, profile mappings, connections, authenticators, I wonder if it makes things clearer to put them under a config bucket .. like /tps/rest/config/audit for example. 9. Is there a mapping between Profile and Profile Mapping operations and the old operations? Please put that in so we can see whether we have complete coverage. In particular, I do not see an operation to approve/enable a profile. This is important because we have Common Criteria requirements that two users are involved in the approval of a profile. In our case, IIRC we implemented it such that an admin can configure a profile but an agent must approve it. 10. In POST /tps/rest/profilemappings, you return the location of the profile mappings as well as the contents of the profile mapping resource. You should just return the location. In fact, its probably better to just return locations in general. The client would then use GET to fetch the details if needed. This comment applies to many of the resources. 11. In the PUT /tps/rest/config, we have requirements for having than one user to approve changes. Admins make changes and agents approve them if I recall correctly. You should look at the differences between the agent and admin pages. On Tue, 2013-06-25 at 13:15 -0500, Endi Sukma Dewata wrote:
Hi,
This is the initial draft of the TPS REST interface design: http://pki.fedoraproject.org/wiki/TPS_REST_Interface
Please take a look and let me know if you have any comments. Thanks!
On 6/27/2013 10:20 AM, Ade Lee wrote:
1. One of the decisions you made was to allow admins, agents and operators to access the same resources. So, unlike the rest of the java subsystems, we will have /tps/rest/tokens as opposed to /tps/rest/agent/tokens or /tps/rest/admin/tokens
One reason one might want to add /agent or /admin is if the content of GET /tps/tokens/X is different for admins/agents/operators.
So what is the difference between op=search and op=search_admin, op=view and op=view_admin, op=show and op=show_admin? etc.
I'm still rebuilding my TPS environment so I can't check this myself right now, but here are the screenshots from the docs and they look pretty much identical: https://access.redhat.com/site/documentation/en-US/Red_Hat_Certificate_Syste... https://access.redhat.com/site/documentation/en-US/Red_Hat_Certificate_Syste... https://access.redhat.com/site/documentation/en-US/Red_Hat_Certificate_Syste...
Is there any case where those differences need be preserved?
Even if there are differences, since they are the same resources (i.e. tokens) most of the visible attributes will be the same. If there are differences between the roles, it will be handled by returning the common attributes plus the attributes visible to that role only. When creating the REST data structure we combine all possible attributes.
2. You use PUT /tps/rest/tokens/X as the operation to modify tokens. Usually REST semantics imply that the resource passed in to the PUT operation essentially replace the resource at the server.
Is that your intention? I'm guessing that the resource that would be passed in is the token object returned in GET /tps/rest/tokens/foo. It would help if you defined what a token resource was.
Any REST resource is a partial representation of the actual object/service running on the server, meaning it may not have the full information to reconstruct the object on the server-side. So even if we PUT a resource it should be clear that we're not completely replacing the actual object on the server, but we're updating the resource with the attributes provided in the request. In case of token resource, the resource consists of the attributes returned by the GET operation which may not include everything a token has. When modifying the resource using PUT some of these attributes will be ignored because they are read-only (e.g. date created/modified), but we can still PUT a resource that we obtained earlier using GET back to the server to update some attributes.
Or is your intention to send in something with an "action" parameter - in which case, the correct operation is a POST?
I use 'action' in the modify token operation because I couldn't find a better replacement for the 'question' parameter in the original op=do_token. Ideally it should be called 'status' but there's already another attribute with that name. Any suggestion? This 'action' should actually be returned by the GET operation too. I'll add it after we finalize the name.
3. Should we be worried about a XSS attack here for modifying the state of the token? My guess is that we need to take advantage of the nonce mechanism, in which case, token state modification will require two steps.
Yes, but similar to our discussion in the past, nonces or ETags can be added in a later phase without changing the interface. Since the modification is a PUT operation, and the request attributes are obtained using a GET operation done earlier, this is a two-step operation. Without the nonce or ETag the GET operation will be optional, and the client can construct the request from scratch. But later the PUT operation can require a nonce or ETag from a previous GET operation.
4. You should also note that we will be sending back BAD_REQUEST (401?) when the token state transition is not permitted.
That should fall under "Normal application errors will return HTTP 4xx return code." We can add more details as we implement it.
5. In the response to PUT /tokens/X, it is not necessary to return the state of the token. Rather, you should return a URL pointer to the newly modified resource. The same comment applies to the other PUT operations as well.
Since PUT operation in general doesn't create a new resource, and also the request is sent to the resource URL being modified, there is no need to return a new location because it will be identical. In this design the new location will only be returned on POST operations when adding a new resource, which in this case the new resource location will be different from the POST target (the resource collection).
6. Same question about XSS for add/remove token/ add/remove user.
Similarly, nonces or ETags can be added later. As described earlier, modifying users is a two-step process too with GET and PUT. Add and remove operations do not require the client to do any other operation, but it can be made so without changing the interface. The DELETE can require a GET, but I'm not sure what operation should be required before add.
7. Note that the difference between op=view_activity, op=view_activity_all etc. is in the types of activities that are visible. This should be handled seamlessly in the logic used to select activity records from the db.
Yes, the interface will be the same, but depending on which roles you belong to you may get different results.
8. For the configuration items - audit config, profiles, profile mappings, connections, authenticators, I wonder if it makes things clearer to put them under a config bucket .. like /tps/rest/config/audit for example.
I'd rather not do that. For now I'm using /tps/rest/config to store general settings. If there are resource-specific settings they should go to that resource subtree. For example, suppose later we want to provide interface to view audit logs, it will be stored under /tps/rest/audit/logs. The audit config can be stored in /tps/rest/audit, or maybe /tps/rest/audit/config. Imagine you're an admin looking at a web page showing the audit logs. If you want to change the audit settings you'd want to be able to click a button on that page directly. Having to go to config -> audit menu will be less intuitive. Not that this is relevant to determine the actual resource URL, but with the same principles the config should be located near the entries of that resource. It's also possible to provide two interfaces at both locations for the same resources, but there's no real need to do that now. For the other resources mentioned above, they are both the entries and the config. If we put them under config there wouldn't many (or any) things left at the top level. Even system users can also be considered a config, but unless there's a strong reason I prefer to keep them where they are now.
9. Is there a mapping between Profile and Profile Mapping operations and the old operations? Please put that in so we can see whether we have complete coverage. In particular, I do not see an operation to approve/enable a profile. This is important because we have Common Criteria requirements that two users are involved in the approval of a profile. In our case, IIRC we implemented it such that an admin can configure a profile but an agent must approve it.
There should be, but they are not in the docs and I have not had a chance to test all possible operations in the UI. Once I get the TPS instance back running I'll continue this. But in general we can implement something similar to the cert requests: /tps/rest/profiles/{id}/approve (all approvers will use the same URL) /tps/rest/profiles/{id}/enable
10. In POST /tps/rest/profilemappings, you return the location of the profile mappings as well as the contents of the profile mapping resource. You should just return the location. In fact, its probably better to just return locations in general. The client would then use GET to fetch the details if needed. This comment applies to many of the resources.
In general I'd rather return the resource attributes in the POST response than requiring the client to do a separate GET later. This response acts as a receipt/acknowledgement for the add request, and it will return the actual attributes stored on the server before any other operation is done on it. This is rather important because sometimes a lazy UI won't do an extra GET because it assumes the attributes don't change, where in fact they could be normalized, ignored, or generated by the server. It also prevents a possible (although unlikely) attack that inserts an operation between the POST and GET.
11. In the PUT /tps/rest/config, we have requirements for having than one user to approve changes. Admins make changes and agents approve them if I recall correctly. You should look at the differences between the agent and admin pages.
I'll take a look again after I have the TPS instance back. Is there any documents or diagrams showing the workflow of all approval processes in TPS UI? Thanks. -- Endi S. Dewata
On Thu, 2013-06-27 at 13:23 -0500, Endi Sukma Dewata wrote:
On 6/27/2013 10:20 AM, Ade Lee wrote:
1. One of the decisions you made was to allow admins, agents and operators to access the same resources. So, unlike the rest of the java subsystems, we will have /tps/rest/tokens as opposed to /tps/rest/agent/tokens or /tps/rest/admin/tokens
One reason one might want to add /agent or /admin is if the content of GET /tps/tokens/X is different for admins/agents/operators.
So what is the difference between op=search and op=search_admin, op=view and op=view_admin, op=show and op=show_admin? etc.
I'm still rebuilding my TPS environment so I can't check this myself right now, but here are the screenshots from the docs and they look pretty much identical:
https://access.redhat.com/site/documentation/en-US/Red_Hat_Certificate_Syste...
https://access.redhat.com/site/documentation/en-US/Red_Hat_Certificate_Syste...
https://access.redhat.com/site/documentation/en-US/Red_Hat_Certificate_Syste...
Is there any case where those differences need be preserved?
Even if there are differences, since they are the same resources (i.e. tokens) most of the visible attributes will be the same. If there are differences between the roles, it will be handled by returning the common attributes plus the attributes visible to that role only. When creating the REST data structure we combine all possible attributes.
I agree that there are more likely than not insufficient differences between the representations of the resources for each role. Part of the reason we chose to keep the calls separate for the other subsystems is because of the different authentication method for each interface. Agent and admin required client auth, ee required no auth. This is not true for the TPS UI - all intefaces require client auth or otherwise.
2. You use PUT /tps/rest/tokens/X as the operation to modify tokens. Usually REST semantics imply that the resource passed in to the PUT operation essentially replace the resource at the server.
Is that your intention? I'm guessing that the resource that would be passed in is the token object returned in GET /tps/rest/tokens/foo. It would help if you defined what a token resource was.
Any REST resource is a partial representation of the actual object/service running on the server, meaning it may not have the full information to reconstruct the object on the server-side. So even if we PUT a resource it should be clear that we're not completely replacing the actual object on the server, but we're updating the resource with the attributes provided in the request.
In case of token resource, the resource consists of the attributes returned by the GET operation which may not include everything a token has. When modifying the resource using PUT some of these attributes will be ignored because they are read-only (e.g. date created/modified), but we can still PUT a resource that we obtained earlier using GET back to the server to update some attributes.
Agreed.
Or is your intention to send in something with an "action" parameter - in which case, the correct operation is a POST?
I use 'action' in the modify token operation because I couldn't find a better replacement for the 'question' parameter in the original op=do_token. Ideally it should be called 'status' but there's already another attribute with that name. Any suggestion? This 'action' should actually be returned by the GET operation too. I'll add it after we finalize the name.
do_token?question=X is essentially an operation where you are attempting to set the state of the token resource - for example, from "active" to "lost". There is no need for an additional parameter.
3. Should we be worried about a XSS attack here for modifying the state of the token? My guess is that we need to take advantage of the nonce mechanism, in which case, token state modification will require two steps.
Yes, but similar to our discussion in the past, nonces or ETags can be added in a later phase without changing the interface. Since the modification is a PUT operation, and the request attributes are obtained using a GET operation done earlier, this is a two-step operation. Without the nonce or ETag the GET operation will be optional, and the client can construct the request from scratch. But later the PUT operation can require a nonce or ETag from a previous GET operation.
OK, please put the nonce in the design. We dont want to forget it.
4. You should also note that we will be sending back BAD_REQUEST (401?) when the token state transition is not permitted.
That should fall under "Normal application errors will return HTTP 4xx return code." We can add more details as we implement it.
Right, I'm just pointing this out because in this case, there is a state machine which determines which operations are successful. From the point of view of the API, doing a token transition is just an attempt to set the state of the token differently, but unlike something simple like just changing an attribute value, the operation is subject to the results of a state machine.
5. In the response to PUT /tokens/X, it is not necessary to return the state of the token. Rather, you should return a URL pointer to the newly modified resource. The same comment applies to the other PUT operations as well.
Since PUT operation in general doesn't create a new resource, and also the request is sent to the resource URL being modified, there is no need to return a new location because it will be identical. In this design the new location will only be returned on POST operations when adding a new resource, which in this case the new resource location will be different from the POST target (the resource collection).
Agreed. But in PUT /tokens/X, you return state information. This is not necessary. You should probably return nothing - other than status.
6. Same question about XSS for add/remove token/ add/remove user.
Similarly, nonces or ETags can be added later. As described earlier, modifying users is a two-step process too with GET and PUT. Add and remove operations do not require the client to do any other operation, but it can be made so without changing the interface. The DELETE can require a GET, but I'm not sure what operation should be required before add.
Yeah - we need to think about ADD. Being able to add a privileged user for example by XSS would be troubling.
7. Note that the difference between op=view_activity, op=view_activity_all etc. is in the types of activities that are visible. This should be handled seamlessly in the logic used to select activity records from the db.
Yes, the interface will be the same, but depending on which roles you belong to you may get different results.
8. For the configuration items - audit config, profiles, profile mappings, connections, authenticators, I wonder if it makes things clearer to put them under a config bucket .. like /tps/rest/config/audit for example.
I'd rather not do that. For now I'm using /tps/rest/config to store general settings. If there are resource-specific settings they should go to that resource subtree.
For example, suppose later we want to provide interface to view audit logs, it will be stored under /tps/rest/audit/logs. The audit config can be stored in /tps/rest/audit, or maybe /tps/rest/audit/config. Imagine you're an admin looking at a web page showing the audit logs. If you want to change the audit settings you'd want to be able to click a button on that page directly. Having to go to config -> audit menu will be less intuitive. Not that this is relevant to determine the actual resource URL, but with the same principles the config should be located near the entries of that resource. It's also possible to provide two interfaces at both locations for the same resources, but there's no real need to do that now.
For the other resources mentioned above, they are both the entries and the config. If we put them under config there wouldn't many (or any) things left at the top level. Even system users can also be considered a config, but unless there's a strong reason I prefer to keep them where they are now.
OK.
9. Is there a mapping between Profile and Profile Mapping operations and the old operations? Please put that in so we can see whether we have complete coverage. In particular, I do not see an operation to approve/enable a profile. This is important because we have Common Criteria requirements that two users are involved in the approval of a profile. In our case, IIRC we implemented it such that an admin can configure a profile but an agent must approve it.
There should be, but they are not in the docs and I have not had a chance to test all possible operations in the UI. Once I get the TPS instance back running I'll continue this. But in general we can implement something similar to the cert requests:
/tps/rest/profiles/{id}/approve (all approvers will use the same URL) /tps/rest/profiles/{id}/enable
Thats fine - lets make sure we get those operations in.
10. In POST /tps/rest/profilemappings, you return the location of the profile mappings as well as the contents of the profile mapping resource. You should just return the location. In fact, its probably better to just return locations in general. The client would then use GET to fetch the details if needed. This comment applies to many of the resources.
In general I'd rather return the resource attributes in the POST response than requiring the client to do a separate GET later. This response acts as a receipt/acknowledgement for the add request, and it will return the actual attributes stored on the server before any other operation is done on it. This is rather important because sometimes a lazy UI won't do an extra GET because it assumes the attributes don't change, where in fact they could be normalized, ignored, or generated by the server. It also prevents a possible (although unlikely) attack that inserts an operation between the POST and GET.
I guess you are suggesting the same for PUT requests as above. I'm not sure whether this is vaid reasoning or not. Just because you see what has been posted (or PUT) in the server does not mean that this data has not been changed the next time the data is accessed. If anything, returning this data encourages lazy clients. The fact is though that we will have to implement nonces and possibly etags, and so GETs will become mandatory for any changes.
11. In the PUT /tps/rest/config, we have requirements for having than one user to approve changes. Admins make changes and agents approve them if I recall correctly. You should look at the differences between the agent and admin pages.
I'll take a look again after I have the TPS instance back. Is there any documents or diagrams showing the workflow of all approval processes in TPS UI? Thanks.
The CC docs are probably the best bet for that (other than looking at the code).
On 6/28/2013 9:36 AM, Ade Lee wrote:
Or is your intention to send in something with an "action" parameter - in which case, the correct operation is a POST?
I use 'action' in the modify token operation because I couldn't find a better replacement for the 'question' parameter in the original op=do_token. Ideally it should be called 'status' but there's already another attribute with that name. Any suggestion? This 'action' should actually be returned by the GET operation too. I'll add it after we finalize the name.
do_token?question=X is essentially an operation where you are attempting to set the state of the token resource - for example, from "active" to "lost". There is no need for an additional parameter.
I added a Change Token State operation: http://pki.fedoraproject.org/wiki/TPS_REST_Interface#Change_Token_State As discussed, for now we'll call this parameter 'next state', which will update the status & reason depending on the current state of the token.
3. Should we be worried about a XSS attack here for modifying the state of the token? My guess is that we need to take advantage of the nonce mechanism, in which case, token state modification will require two steps.
Yes, but similar to our discussion in the past, nonces or ETags can be added in a later phase without changing the interface. Since the modification is a PUT operation, and the request attributes are obtained using a GET operation done earlier, this is a two-step operation. Without the nonce or ETag the GET operation will be optional, and the client can construct the request from scratch. But later the PUT operation can require a nonce or ETag from a previous GET operation.
OK, please put the nonce in the design. We dont want to forget it.
I added two sections related to this: http://pki.fedoraproject.org/wiki/TPS_REST_Interface#Concurrency_Control http://pki.fedoraproject.org/wiki/TPS_REST_Interface#Vulnerabilities
4. You should also note that we will be sending back BAD_REQUEST (401?) when the token state transition is not permitted.
That should fall under "Normal application errors will return HTTP 4xx return code." We can add more details as we implement it.
Right, I'm just pointing this out because in this case, there is a state machine which determines which operations are successful. From the point of view of the API, doing a token transition is just an attempt to set the state of the token differently, but unlike something simple like just changing an attribute value, the operation is subject to the results of a state machine.
I noted that in the response of Change Token State: http://pki.fedoraproject.org/wiki/TPS_REST_Interface#Change_Token_State
5. In the response to PUT /tokens/X, it is not necessary to return the state of the token. Rather, you should return a URL pointer to the newly modified resource. The same comment applies to the other PUT operations as well.
Since PUT operation in general doesn't create a new resource, and also the request is sent to the resource URL being modified, there is no need to return a new location because it will be identical. In this design the new location will only be returned on POST operations when adding a new resource, which in this case the new resource location will be different from the POST target (the resource collection).
Agreed. But in PUT /tokens/X, you return state information. This is not necessary. You should probably return nothing - other than status.
See explanation in #10.
6. Same question about XSS for add/remove token/ add/remove user.
Similarly, nonces or ETags can be added later. As described earlier, modifying users is a two-step process too with GET and PUT. Add and remove operations do not require the client to do any other operation, but it can be made so without changing the interface. The DELETE can require a GET, but I'm not sure what operation should be required before add.
Yeah - we need to think about ADD. Being able to add a privileged user for example by XSS would be troubling.
I think we're mixing up CSRF and XSS. Nonces will be useful to prevent CSRF. XSS can be prevented by encoding/escaping the values. If we implement a single-page Web UI (like IPA Web UI) the client can obtain a nonce during login, then the UI will keep the nonce in memory throughout the session and use it for all update requests including add (the nonce should really be called CSRF token). CSRF attack usually will open a new page, so it will not be able to know the nonce. If we implement a multi-page Web UI (like the current Dogtag UI), the add operation can be done in 2 steps. The first page will generate the nonce, then the second page will execute the operation.
9. Is there a mapping between Profile and Profile Mapping operations and the old operations? Please put that in so we can see whether we have complete coverage. In particular, I do not see an operation to approve/enable a profile. This is important because we have Common Criteria requirements that two users are involved in the approval of a profile. In our case, IIRC we implemented it such that an admin can configure a profile but an agent must approve it.
There should be, but they are not in the docs and I have not had a chance to test all possible operations in the UI. Once I get the TPS instance back running I'll continue this. But in general we can implement something similar to the cert requests:
/tps/rest/profiles/{id}/approve (all approvers will use the same URL) /tps/rest/profiles/{id}/enable
Thats fine - lets make sure we get those operations in.
I added the missing mappings. I also added a Change Profile State operation: http://pki.fedoraproject.org/wiki/TPS_REST_Interface#Change_Profile_State
10. In POST /tps/rest/profilemappings, you return the location of the profile mappings as well as the contents of the profile mapping resource. You should just return the location. In fact, its probably better to just return locations in general. The client would then use GET to fetch the details if needed. This comment applies to many of the resources.
In general I'd rather return the resource attributes in the POST response than requiring the client to do a separate GET later. This response acts as a receipt/acknowledgement for the add request, and it will return the actual attributes stored on the server before any other operation is done on it. This is rather important because sometimes a lazy UI won't do an extra GET because it assumes the attributes don't change, where in fact they could be normalized, ignored, or generated by the server. It also prevents a possible (although unlikely) attack that inserts an operation between the POST and GET.
I guess you are suggesting the same for PUT requests as above. I'm not sure whether this is vaid reasoning or not. Just because you see what has been posted (or PUT) in the server does not mean that this data has not been changed the next time the data is accessed.
There's never any guarantee that the data we get (either from PUT, POST, or even GET) has not been changed when we submit it for another update operation. The advantage of returning the data in PUT/POST response is we're pushing the updated data to the client's feet.
If anything, returning this data encourages lazy clients.
On the contrary, by returning the data in the PUT/POST response the client will have little reason not to use it. If the client has no intention to get the new data anyway (either from PUT/POST response or by doing another GET) there's nothing we can do about it. But if the client wants to get the new data, using PUT/POST response is a much cheaper option. Compare the following code: user = users.update(user); with this: users.update(user); user = users.get(user.getId());
The fact is though that we will have to implement nonces and possibly etags, and so GETs will become mandatory for any changes.
Not necessarily. ETag is only an optimistic lock; it's not a guarantee that the data won't change. The PUT/POST response will also provide ETag. Also see comment #6, we may be able to reuse the nonce. If there's nobody else updating the same data, we could reuse the PUT/POST response and the ETag that came with it to make subsequent update. So we're still using a two-step update process, but the 2nd step of the first update may become the 1st step of the second update. See also the Concurrency Control section in the wiki page. Imagine this, you updated a user, then you went to do some other operations. Then you came back to this user and edited it again. If there's nobody else changing the user, then the new data and ETag that you got from the first update would still be valid. But if first update happened a long time ago, the client may decide according to the caching policy that the data has expired and then issue a GET to get a newer data and ETag.
11. In the PUT /tps/rest/config, we have requirements for having than one user to approve changes. Admins make changes and agents approve them if I recall correctly. You should look at the differences between the agent and admin pages.
I'll take a look again after I have the TPS instance back. Is there any documents or diagrams showing the workflow of all approval processes in TPS UI? Thanks.
The CC docs are probably the best bet for that (other than looking at the code).
The only approval process I saw in the UI is for profiles, not for general config. Could you point me to a specific page in the docs? Thanks. -- Endi S. Dewata
I'll respond to the comments below in a separate email. One thing I just noticed though is the operation to add a token. You have POST /tokens passing in the tokenId. POST /tokens is only used when creating a resource when the URL of the resource is unknown -- ie. it will be specified by the server. An example of this is POST /certrequests which returns the URL of the request /certrequests/0xf00 where 0xf00 is the request ID as assigned by the server. In this case, the token ID is what is being passed in. So we know the resultant URL when the resource is created. There are two options instead: a) PUT /tokens/tokenID b) POST /tokens/tokenID My preference is a) but of course the advantage of (b) is that there is a distinction between an ADD and a modify operation. The same comment applies to all the ADD operations in the design. On Fri, 2013-06-28 at 23:57 -0500, Endi Sukma Dewata wrote:
On 6/28/2013 9:36 AM, Ade Lee wrote:
Or is your intention to send in something with an "action" parameter - in which case, the correct operation is a POST?
I use 'action' in the modify token operation because I couldn't find a better replacement for the 'question' parameter in the original op=do_token. Ideally it should be called 'status' but there's already another attribute with that name. Any suggestion? This 'action' should actually be returned by the GET operation too. I'll add it after we finalize the name.
do_token?question=X is essentially an operation where you are attempting to set the state of the token resource - for example, from "active" to "lost". There is no need for an additional parameter.
I added a Change Token State operation: http://pki.fedoraproject.org/wiki/TPS_REST_Interface#Change_Token_State
As discussed, for now we'll call this parameter 'next state', which will update the status & reason depending on the current state of the token.
3. Should we be worried about a XSS attack here for modifying the state of the token? My guess is that we need to take advantage of the nonce mechanism, in which case, token state modification will require two steps.
Yes, but similar to our discussion in the past, nonces or ETags can be added in a later phase without changing the interface. Since the modification is a PUT operation, and the request attributes are obtained using a GET operation done earlier, this is a two-step operation. Without the nonce or ETag the GET operation will be optional, and the client can construct the request from scratch. But later the PUT operation can require a nonce or ETag from a previous GET operation.
OK, please put the nonce in the design. We dont want to forget it.
I added two sections related to this: http://pki.fedoraproject.org/wiki/TPS_REST_Interface#Concurrency_Control http://pki.fedoraproject.org/wiki/TPS_REST_Interface#Vulnerabilities
4. You should also note that we will be sending back BAD_REQUEST (401?) when the token state transition is not permitted.
That should fall under "Normal application errors will return HTTP 4xx return code." We can add more details as we implement it.
Right, I'm just pointing this out because in this case, there is a state machine which determines which operations are successful. From the point of view of the API, doing a token transition is just an attempt to set the state of the token differently, but unlike something simple like just changing an attribute value, the operation is subject to the results of a state machine.
I noted that in the response of Change Token State: http://pki.fedoraproject.org/wiki/TPS_REST_Interface#Change_Token_State
5. In the response to PUT /tokens/X, it is not necessary to return the state of the token. Rather, you should return a URL pointer to the newly modified resource. The same comment applies to the other PUT operations as well.
Since PUT operation in general doesn't create a new resource, and also the request is sent to the resource URL being modified, there is no need to return a new location because it will be identical. In this design the new location will only be returned on POST operations when adding a new resource, which in this case the new resource location will be different from the POST target (the resource collection).
Agreed. But in PUT /tokens/X, you return state information. This is not necessary. You should probably return nothing - other than status.
See explanation in #10.
6. Same question about XSS for add/remove token/ add/remove user.
Similarly, nonces or ETags can be added later. As described earlier, modifying users is a two-step process too with GET and PUT. Add and remove operations do not require the client to do any other operation, but it can be made so without changing the interface. The DELETE can require a GET, but I'm not sure what operation should be required before add.
Yeah - we need to think about ADD. Being able to add a privileged user for example by XSS would be troubling.
I think we're mixing up CSRF and XSS. Nonces will be useful to prevent CSRF. XSS can be prevented by encoding/escaping the values.
If we implement a single-page Web UI (like IPA Web UI) the client can obtain a nonce during login, then the UI will keep the nonce in memory throughout the session and use it for all update requests including add (the nonce should really be called CSRF token). CSRF attack usually will open a new page, so it will not be able to know the nonce.
If we implement a multi-page Web UI (like the current Dogtag UI), the add operation can be done in 2 steps. The first page will generate the nonce, then the second page will execute the operation.
9. Is there a mapping between Profile and Profile Mapping operations and the old operations? Please put that in so we can see whether we have complete coverage. In particular, I do not see an operation to approve/enable a profile. This is important because we have Common Criteria requirements that two users are involved in the approval of a profile. In our case, IIRC we implemented it such that an admin can configure a profile but an agent must approve it.
There should be, but they are not in the docs and I have not had a chance to test all possible operations in the UI. Once I get the TPS instance back running I'll continue this. But in general we can implement something similar to the cert requests:
/tps/rest/profiles/{id}/approve (all approvers will use the same URL) /tps/rest/profiles/{id}/enable
Thats fine - lets make sure we get those operations in.
I added the missing mappings. I also added a Change Profile State operation: http://pki.fedoraproject.org/wiki/TPS_REST_Interface#Change_Profile_State
10. In POST /tps/rest/profilemappings, you return the location of the profile mappings as well as the contents of the profile mapping resource. You should just return the location. In fact, its probably better to just return locations in general. The client would then use GET to fetch the details if needed. This comment applies to many of the resources.
In general I'd rather return the resource attributes in the POST response than requiring the client to do a separate GET later. This response acts as a receipt/acknowledgement for the add request, and it will return the actual attributes stored on the server before any other operation is done on it. This is rather important because sometimes a lazy UI won't do an extra GET because it assumes the attributes don't change, where in fact they could be normalized, ignored, or generated by the server. It also prevents a possible (although unlikely) attack that inserts an operation between the POST and GET.
I guess you are suggesting the same for PUT requests as above. I'm not sure whether this is vaid reasoning or not. Just because you see what has been posted (or PUT) in the server does not mean that this data has not been changed the next time the data is accessed.
There's never any guarantee that the data we get (either from PUT, POST, or even GET) has not been changed when we submit it for another update operation. The advantage of returning the data in PUT/POST response is we're pushing the updated data to the client's feet.
If anything, returning this data encourages lazy clients.
On the contrary, by returning the data in the PUT/POST response the client will have little reason not to use it. If the client has no intention to get the new data anyway (either from PUT/POST response or by doing another GET) there's nothing we can do about it. But if the client wants to get the new data, using PUT/POST response is a much cheaper option.
Compare the following code:
user = users.update(user);
with this:
users.update(user); user = users.get(user.getId());
The fact is though that we will have to implement nonces and possibly etags, and so GETs will become mandatory for any changes.
Not necessarily. ETag is only an optimistic lock; it's not a guarantee that the data won't change. The PUT/POST response will also provide ETag. Also see comment #6, we may be able to reuse the nonce.
If there's nobody else updating the same data, we could reuse the PUT/POST response and the ETag that came with it to make subsequent update. So we're still using a two-step update process, but the 2nd step of the first update may become the 1st step of the second update. See also the Concurrency Control section in the wiki page.
Imagine this, you updated a user, then you went to do some other operations. Then you came back to this user and edited it again. If there's nobody else changing the user, then the new data and ETag that you got from the first update would still be valid. But if first update happened a long time ago, the client may decide according to the caching policy that the data has expired and then issue a GET to get a newer data and ETag.
11. In the PUT /tps/rest/config, we have requirements for having than one user to approve changes. Admins make changes and agents approve them if I recall correctly. You should look at the differences between the agent and admin pages.
I'll take a look again after I have the TPS instance back. Is there any documents or diagrams showing the workflow of all approval processes in TPS UI? Thanks.
The CC docs are probably the best bet for that (other than looking at the code).
The only approval process I saw in the UI is for profiles, not for general config. Could you point me to a specific page in the docs?
Thanks.
On 6/29/2013 2:16 PM, Ade Lee wrote:
One thing I just noticed though is the operation to add a token. You have POST /tokens passing in the tokenId.
POST /tokens is only used when creating a resource when the URL of the resource is unknown -- ie. it will be specified by the server. An example of this is POST /certrequests which returns the URL of the request /certrequests/0xf00 where 0xf00 is the request ID as assigned by the server.
That is one way to use POST, but in general it doesn't always have to be that limited. Consider the ID as an optional parameter. You can specify it, but it's not specified the server could generate a new ID automatically (not that we want to do that for this case). For the client it would be easier to use the same operation to create a new entry, with or without the ID. The operation that will work for both cases is POST-ing into the collection. I don't think this is violating any REST guidelines. It's also consistent with the add operation for the existing user and group resources.
In this case, the token ID is what is being passed in. So we know the resultant URL when the resource is created. There are two options instead:
a) PUT /tokens/tokenID b) POST /tokens/tokenID
My preference is a) but of course the advantage of (b) is that there is a distinction between an ADD and a modify operation.
The same comment applies to all the ADD operations in the design.
The problem with (a) is that without ETag there's a risk overwriting an existing token, and there's no warning or error message when that happens. While ETag is desired, it should be optional (can we guarantee all clients will support ETag?) and may not get implemented right away. If we POST to the collection the server can reject it if the entry already exists. Also, naturally an add operation is not idempotent, so we should not use PUT. Option (b) is a little weird because usually we expect the resource would already exist when we POST. POST-ing to the collection is fine because the collection always exists. -- Endi S. Dewata
On Sat, 2013-06-29 at 16:12 -0500, Endi Sukma Dewata wrote:
On 6/29/2013 2:16 PM, Ade Lee wrote:
One thing I just noticed though is the operation to add a token. You have POST /tokens passing in the tokenId.
POST /tokens is only used when creating a resource when the URL of the resource is unknown -- ie. it will be specified by the server. An example of this is POST /certrequests which returns the URL of the request /certrequests/0xf00 where 0xf00 is the request ID as assigned by the server.
That is one way to use POST, but in general it doesn't always have to be that limited. Consider the ID as an optional parameter. You can specify it, but it's not specified the server could generate a new ID automatically (not that we want to do that for this case). For the client it would be easier to use the same operation to create a new entry, with or without the ID. The operation that will work for both cases is POST-ing into the collection.
I don't think this is violating any REST guidelines. It's also consistent with the add operation for the existing user and group resources.
Maybe we need to revisit these then.
In this case, the token ID is what is being passed in. So we know the resultant URL when the resource is created. There are two options instead:
a) PUT /tokens/tokenID b) POST /tokens/tokenID
My preference is a) but of course the advantage of (b) is that there is a distinction between an ADD and a modify operation.
The same comment applies to all the ADD operations in the design.
The problem with (a) is that without ETag there's a risk overwriting an existing token, and there's no warning or error message when that happens. While ETag is desired, it should be optional (can we guarantee all clients will support ETag?) and may not get implemented right away. If we POST to the collection the server can reject it if the entry already exists. Also, naturally an add operation is not idempotent, so we should not use PUT.
The REST guidelines are precisely that - guidelines - and we can choose to ignore them at our discretion. But there are distinct advantages to following them. In particular, when doing so, anyone looking at the API knows exactly what is going on. By using PUT, its clear that the client is responsible for providing the resource ID. REST basically says to use the HTTP spec to determine what operation to use. To see what the REST guideline would be in this case, lets see what the HTTP spec says about PUT: http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.6 The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI. If a new resource is created, the origin server MUST inform the user agent via the 201 (Created) response. If an existing resource is modified, either the 200 (OK) or 204 (No Content) response codes SHOULD be sent to indicate successful completion of the request. The point you make about possibly overwriting an existing resource is valid -- but the answer there is etags. And as e will be writing the first clients for this, we can safely say that - yes - all clients will support etags. Just because we may not implement etags in the first iteration does not mean that we should use non-standard API in our design.
Option (b) is a little weird because usually we expect the resource would already exist when we POST. POST-ing to the collection is fine because the collection always exists.
I agree that option (b) is weird.
On 7/1/2013 9:47 AM, Ade Lee wrote:
The REST guidelines are precisely that - guidelines - and we can choose to ignore them at our discretion. But there are distinct advantages to following them. In particular, when doing so, anyone looking at the API knows exactly what is going on. By using PUT, its clear that the client is responsible for providing the resource ID.
REST basically says to use the HTTP spec to determine what operation to use. To see what the REST guideline would be in this case, lets see what the HTTP spec says about PUT: http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.6
The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI. If a new resource is created, the origin server MUST inform the user agent via the 201 (Created) response. If an existing resource is modified, either the 200 (OK) or 204 (No Content) response codes SHOULD be sent to indicate successful completion of the request.
This defines what PUT does, but it doesn't necessarily require us to use PUT if we know the resource ID already. The POST is equally valid: From http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.5:
The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line. POST is designed to allow a uniform method to cover the following functions:
- Annotation of existing resources; - Posting a message to a bulletin board, newsgroup, mailing list, or similar group of articles; - Providing a block of data, such as the result of submitting a form, to a data-handling process; - Extending a database through an append operation.
The actual function performed by the POST method is determined by the server and is usually dependent on the Request-URI. The posted entity is subordinate to that URI in the same way that a file is subordinate to a directory containing it, a news article is subordinate to a newsgroup to which it is posted, or a record is subordinate to a database.
Adding a new user or profile into a collection would be analogous to the above examples. On 7/1/2013 9:47 AM, Ade Lee wrote:
The point you make about possibly overwriting an existing resource is valid -- but the answer there is etags. And as e will be writing the first clients for this, we can safely say that - yes - all clients will support etags.
With REST anybody can write their own client or use generic tools to access our interface. Of course the client must provide the required parameters in all requests, but if we require clients to use what supposed to be optional in REST we will reduce the number of clients that can use the interface. REST provides two different -- but valid -- mechanisms: PUT and POST. I think there are enough reasons to use separate operations for create and update: * Different behavior: create should fail if entry already exists where as update should fail if entry doesn't exist. * Different security: not all users are allowed to create an entry. * Server won't need to check if entry already exists. * If operation fails because entry exists/doesn't exist the client will know it. * Client won't need to determine which URL to use. * It works without ETag. Client won't need to do an extra GET operation. * Client can still use ETag if concurrency is a concern. Here's what Bill Burke said when we discussed about this when we're discussing the users interface: On 5/24/2012 11:33 AM, Bill Burke wrote:
1. Considering all the requirements, should CREATE and UPDATE be handled by a single or separate methods?
Because you only want admins to be able to create a user, and non-admins can update users, you should probably use the @Post method to create, and not use @Put for creation.
@POST @Path("/users") create();
for
@PUT @Path("/users/{id}") update(); you should throw a 404 if the user doesn't exist.
Suppose the security requirements are the same for both operations, would you design it differently?
All depends whether or not I wanted to rely on the client to pick the URI or not. I usually use the @POST /users because it gives the server a lot of flexibility on what the returned URI (in the Location header) looks like.
On 7/1/2013 9:47 AM, Ade Lee wrote:
Just because we may not implement etags in the first iteration does not mean that we should use non-standard API in our design.
Using POST won't be non-standard, but requiring ETag for this to work might be. Here's what Bill said:
3. Is it OK to require the use of ETag to distinguish CREATE and UPDATE (requiring a GET operation before every UPDATE)? Or should ETag be optional and used only if the system requires caching or concurrency control?
It is never OK to override the meaning of a header, HTTP method, or response code. One of the nice things about REST is that you have a solid foundation of how a service behaves without ever looking at the documentation of that service: This is because you know the HTTP protocol.
ETag should only be used for conditional GET (when a cache is involved) or a conditional PUT, or POST (When concurrency is involved).
To me this means if someone installs a system and there's only one client using it, or any other situation where concurrency is not an issue, he shouldn't be required to use ETag. -- Endi S. Dewata
On 7/1/2013 11:51 AM, Endi Sukma Dewata wrote:
From http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.5:
The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line. POST is designed to allow a uniform method to cover the following functions:
- Annotation of existing resources; - Posting a message to a bulletin board, newsgroup, mailing list, or similar group of articles; - Providing a block of data, such as the result of submitting a form, to a data-handling process; - Extending a database through an append operation.
The actual function performed by the POST method is determined by the server and is usually dependent on the Request-URI. The posted entity is subordinate to that URI in the same way that a file is subordinate to a directory containing it, a news article is subordinate to a newsgroup to which it is posted, or a record is subordinate to a database.
Adding a new user or profile into a collection would be analogous to the above examples.
Just to clarify, when adding a file into a directory we know the file name already. Still, this is given as an example of a POST operation. So, knowing the ID ahead of time is irrelevant to picking PUT vs. POST to create a resource. -- Endi S. Dewata
On Mon, 2013-07-01 at 12:30 -0500, Endi Sukma Dewata wrote:
On 7/1/2013 11:51 AM, Endi Sukma Dewata wrote:
From http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.5:
The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line. POST is designed to allow a uniform method to cover the following functions:
- Annotation of existing resources; - Posting a message to a bulletin board, newsgroup, mailing list, or similar group of articles; - Providing a block of data, such as the result of submitting a form, to a data-handling process; - Extending a database through an append operation.
The actual function performed by the POST method is determined by the server and is usually dependent on the Request-URI. The posted entity is subordinate to that URI in the same way that a file is subordinate to a directory containing it, a news article is subordinate to a newsgroup to which it is posted, or a record is subordinate to a database.
Adding a new user or profile into a collection would be analogous to the above examples.
Just to clarify, when adding a file into a directory we know the file name already. Still, this is given as an example of a POST operation. So, knowing the ID ahead of time is irrelevant to picking PUT vs. POST to create a resource.
Just to be clear, it is perfectly RESTful to create an interface using only GET and POST. POST can literally be used for pretty much anything and still be restful. The question is more one of restful practice. Its far more intuitive and friendly to use the other verbs PUT and DELETE, even though there may be some clients that do not support these operations. As to this specific use of PUT vs. GET, I've seen best practice guidelines that advocate both scenarios. I am, however, most convinced by the idea that the POST allows the server more flexibility in selecting the Location. An example I saw of using PUT was for a user being able to store their personal set of favorites. For this case, it is entirely appropriate that the user controls the location URL. In our case, what we are creating are server resources and we want control over that. So, point conceded.
On Fri, 2013-06-28 at 23:57 -0500, Endi Sukma Dewata wrote:
On 6/28/2013 9:36 AM, Ade Lee wrote:
Or is your intention to send in something with an "action" parameter - in which case, the correct operation is a POST?
I use 'action' in the modify token operation because I couldn't find a better replacement for the 'question' parameter in the original op=do_token. Ideally it should be called 'status' but there's already another attribute with that name. Any suggestion? This 'action' should actually be returned by the GET operation too. I'll add it after we finalize the name.
do_token?question=X is essentially an operation where you are attempting to set the state of the token resource - for example, from "active" to "lost". There is no need for an additional parameter.
I added a Change Token State operation: http://pki.fedoraproject.org/wiki/TPS_REST_Interface#Change_Token_State
As discussed, for now we'll call this parameter 'next state', which will update the status & reason depending on the current state of the token.
3. Should we be worried about a XSS attack here for modifying the state of the token? My guess is that we need to take advantage of the nonce mechanism, in which case, token state modification will require two steps.
Yes, but similar to our discussion in the past, nonces or ETags can be added in a later phase without changing the interface. Since the modification is a PUT operation, and the request attributes are obtained using a GET operation done earlier, this is a two-step operation. Without the nonce or ETag the GET operation will be optional, and the client can construct the request from scratch. But later the PUT operation can require a nonce or ETag from a previous GET operation.
OK, please put the nonce in the design. We dont want to forget it.
I added two sections related to this: http://pki.fedoraproject.org/wiki/TPS_REST_Interface#Concurrency_Control http://pki.fedoraproject.org/wiki/TPS_REST_Interface#Vulnerabilities
4. You should also note that we will be sending back BAD_REQUEST (401?) when the token state transition is not permitted.
That should fall under "Normal application errors will return HTTP 4xx return code." We can add more details as we implement it.
Right, I'm just pointing this out because in this case, there is a state machine which determines which operations are successful. From the point of view of the API, doing a token transition is just an attempt to set the state of the token differently, but unlike something simple like just changing an attribute value, the operation is subject to the results of a state machine.
I noted that in the response of Change Token State: http://pki.fedoraproject.org/wiki/TPS_REST_Interface#Change_Token_State
5. In the response to PUT /tokens/X, it is not necessary to return the state of the token. Rather, you should return a URL pointer to the newly modified resource. The same comment applies to the other PUT operations as well.
Since PUT operation in general doesn't create a new resource, and also the request is sent to the resource URL being modified, there is no need to return a new location because it will be identical. In this design the new location will only be returned on POST operations when adding a new resource, which in this case the new resource location will be different from the POST target (the resource collection).
Agreed. But in PUT /tokens/X, you return state information. This is not necessary. You should probably return nothing - other than status.
See explanation in #10.
6. Same question about XSS for add/remove token/ add/remove user.
Similarly, nonces or ETags can be added later. As described earlier, modifying users is a two-step process too with GET and PUT. Add and remove operations do not require the client to do any other operation, but it can be made so without changing the interface. The DELETE can require a GET, but I'm not sure what operation should be required before add.
Yeah - we need to think about ADD. Being able to add a privileged user for example by XSS would be troubling.
I think we're mixing up CSRF and XSS. Nonces will be useful to prevent CSRF. XSS can be prevented by encoding/escaping the values.
Thats correct. I was thinking about CSRF.
If we implement a single-page Web UI (like IPA Web UI) the client can obtain a nonce during login, then the UI will keep the nonce in memory throughout the session and use it for all update requests including add (the nonce should really be called CSRF token). CSRF attack usually will open a new page, so it will not be able to know the nonce.
If we implement a multi-page Web UI (like the current Dogtag UI), the add operation can be done in 2 steps. The first page will generate the nonce, then the second page will execute the operation.
9. Is there a mapping between Profile and Profile Mapping operations and the old operations? Please put that in so we can see whether we have complete coverage. In particular, I do not see an operation to approve/enable a profile. This is important because we have Common Criteria requirements that two users are involved in the approval of a profile. In our case, IIRC we implemented it such that an admin can configure a profile but an agent must approve it.
There should be, but they are not in the docs and I have not had a chance to test all possible operations in the UI. Once I get the TPS instance back running I'll continue this. But in general we can implement something similar to the cert requests:
/tps/rest/profiles/{id}/approve (all approvers will use the same URL) /tps/rest/profiles/{id}/enable
Thats fine - lets make sure we get those operations in.
I added the missing mappings. I also added a Change Profile State operation: http://pki.fedoraproject.org/wiki/TPS_REST_Interface#Change_Profile_State
The way this is written, it looks like you are just doing a POST to the /tps/rest/profiles/<ProfileID> and passing in "action" as a parameter. Thats not very RESTful at all. I think we want: /tps/rest/profiles/{id}/{action} where {action} is approve etc. This is also consistent with how we have done this for cert-requests etc. as well.
10. In POST /tps/rest/profilemappings, you return the location of the profile mappings as well as the contents of the profile mapping resource. You should just return the location. In fact, its probably better to just return locations in general. The client would then use GET to fetch the details if needed. This comment applies to many of the resources.
In general I'd rather return the resource attributes in the POST response than requiring the client to do a separate GET later. This response acts as a receipt/acknowledgement for the add request, and it will return the actual attributes stored on the server before any other operation is done on it. This is rather important because sometimes a lazy UI won't do an extra GET because it assumes the attributes don't change, where in fact they could be normalized, ignored, or generated by the server. It also prevents a possible (although unlikely) attack that inserts an operation between the POST and GET.
I guess you are suggesting the same for PUT requests as above. I'm not sure whether this is vaid reasoning or not. Just because you see what has been posted (or PUT) in the server does not mean that this data has not been changed the next time the data is accessed.
There's never any guarantee that the data we get (either from PUT, POST, or even GET) has not been changed when we submit it for another update operation. The advantage of returning the data in PUT/POST response is we're pushing the updated data to the client's feet.
If anything, returning this data encourages lazy clients.
On the contrary, by returning the data in the PUT/POST response the client will have little reason not to use it. If the client has no intention to get the new data anyway (either from PUT/POST response or by doing another GET) there's nothing we can do about it. But if the client wants to get the new data, using PUT/POST response is a much cheaper option.
Compare the following code:
user = users.update(user);
with this:
users.update(user); user = users.get(user.getId());
The fact is though that we will have to implement nonces and possibly etags, and so GETs will become mandatory for any changes.
Not necessarily. ETag is only an optimistic lock; it's not a guarantee that the data won't change. The PUT/POST response will also provide ETag. Also see comment #6, we may be able to reuse the nonce.
If there's nobody else updating the same data, we could reuse the PUT/POST response and the ETag that came with it to make subsequent update. So we're still using a two-step update process, but the 2nd step of the first update may become the 1st step of the second update. See also the Concurrency Control section in the wiki page.
Imagine this, you updated a user, then you went to do some other operations. Then you came back to this user and edited it again. If there's nobody else changing the user, then the new data and ETag that you got from the first update would still be valid. But if first update happened a long time ago, the client may decide according to the caching policy that the data has expired and then issue a GET to get a newer data and ETag.
OK - I like the idea that the etag returned by the PUT/POST can be used for subsequent updates. We can go with this for now.
11. In the PUT /tps/rest/config, we have requirements for having than one user to approve changes. Admins make changes and agents approve them if I recall correctly. You should look at the differences between the agent and admin pages.
I'll take a look again after I have the TPS instance back. Is there any documents or diagrams showing the workflow of all approval processes in TPS UI? Thanks.
The CC docs are probably the best bet for that (other than looking at the code).
The only approval process I saw in the UI is for profiles, not for general config. Could you point me to a specific page in the docs?
Yeah - I think thats right actually. All of the stuff thats in config did not require two agent approval.
Thanks.
On 7/1/2013 11:37 AM, Ade Lee wrote:
I also added a Change Profile State operation: http://pki.fedoraproject.org/wiki/TPS_REST_Interface#Change_Profile_State
The way this is written, it looks like you are just doing a POST to the /tps/rest/profiles/<ProfileID> and passing in "action" as a parameter. Thats not very RESTful at all.
I think we want: /tps/rest/profiles/{id}/{action}
where {action} is approve etc. This is also consistent with how we have done this for cert-requests etc. as well.
This is where I'd like to see the current API changed. These actions (e.g. approve, enable) are not resources or collections. None of the other operations (GET, PUT, DELETE) make sense on actions. On the other hand, using POST to process data is a valid & RESTful operation. POST request should be sent to a resource, and the resource that we want to handle these actions is the target of the action (i.e. profile). According to POST definition: * The actual function performed by the POST method is determined by the server and is usually dependent on the Request-URI. * The action performed by the POST method might not result in a resource that can be identified by a URI. * Example: - Providing a block of data, such as the result of submitting a form, to a data-handling process; According to http://www.ietf.org/rfc/rfc2616.txt (9.6 PUT):
The URI in a POST request identifies the resource that will handle the enclosed entity. That resource might be a data-accepting process, a gateway to some other protocol, or a separate entity that accepts annotations.
Also, POST-ing to /tps/rest/profiles/{id} will be more future-proof in case we add/remove actions later. -- Endi S. Dewata
On Mon, 2013-07-01 at 13:12 -0500, Endi Sukma Dewata wrote:
On 7/1/2013 11:37 AM, Ade Lee wrote:
I also added a Change Profile State operation: http://pki.fedoraproject.org/wiki/TPS_REST_Interface#Change_Profile_State
The way this is written, it looks like you are just doing a POST to the /tps/rest/profiles/<ProfileID> and passing in "action" as a parameter. Thats not very RESTful at all.
I think we want: /tps/rest/profiles/{id}/{action}
where {action} is approve etc. This is also consistent with how we have done this for cert-requests etc. as well.
This is where I'd like to see the current API changed. These actions (e.g. approve, enable) are not resources or collections. None of the other operations (GET, PUT, DELETE) make sense on actions.
On the other hand, using POST to process data is a valid & RESTful operation. POST request should be sent to a resource, and the resource that we want to handle these actions is the target of the action (i.e. profile).
According to POST definition: * The actual function performed by the POST method is determined by the server and is usually dependent on the Request-URI. * The action performed by the POST method might not result in a resource that can be identified by a URI. * Example: - Providing a block of data, such as the result of submitting a form, to a data-handling process;
According to http://www.ietf.org/rfc/rfc2616.txt (9.6 PUT):
The URI in a POST request identifies the resource that will handle the enclosed entity. That resource might be a data-accepting process, a gateway to some other protocol, or a separate entity that accepts annotations.
Endi, the above definition is what has been referred to as overloaded-POST. Basically, its the way the whole RPC world uses the web. It would be entirely RESTful to create an interface using only POST - but wouldn't exactly be a Restful Oriented Architecture (ROA). Similarly one could create an entire interface using just GET and POST, entirely RESTful but not ROA. I looked back into my REST resources - in particular, Restful Web Services by Leonard Richardson and Sam Ruby, to find some standard practices/ advice to counter this approach. What I found was in fact the opposite. Let me quote Richardson: URIs are supposed to designate resources, not operations on the resources. This means it’s almost never appropriate to put the names of operations in your URIs. If you have a URI that looks like /object/do-operation, you’re in danger of slipping into the RPC style. Nobody wants to link to do-operation: they want to link to the object. Expose the operation through the uniform interface, or use overloaded POST if you have to, but make your URIs designate objects, not operations on the objects. So according to Richardson at least, its more RESTful to use overloaded POST and a query variable then to do what we have been doing.
Also, POST-ing to /tps/rest/profiles/{id} will be more future-proof in case we add/remove actions later.
Yes, this is a good point, although I can't shake the feeling that this makes the interface less intuitive and less discoverable. The actions we have exposed as for example, POST /certrequest/{id}/approve are basic operations that we currently expose and will want to expose in the future. POST /certrequest/{id} with action=approve seems more RPC-like in some way. Maybe its because the above mechanism really is RPC-like (because it uses overloaded POST) and using the action variable just makes that more apparent. Or maybe I need to shake the habit of creating URLs. If we decide to do this, we should consider changing the existing interfaces to follow this practice, before too many clients are written. Incidentally, there is a case where POST /foo/operation is acceptable and that is where you are in fact providing a processor to do an operation and are not generating resources. An example of this is in the TKS, where we request generation of a session key to communicate with a token.
participants (2)
-
Ade Lee -
Endi Sukma Dewata